Inter-coder reliability of categorising force-dynamic events in human-technology interaction
Zitieren Sie bitte immer diese URN: urn:nbn:de:bvb:20-opus-194127
- Two studies are reported that investigate how readily accessible and applicable ten force-dynamic categories are to novices in describing short episodes of human-technology interaction (Study 1) and that establish a measure of inter-coder reliability when re-classifying these episodes into force-dynamic categories (Study 2). The results of the first study show that people can easily and confidently relate their experiences with technology to the definitions of force-dynamic events (e.g. “The driver released the handbrake” as an example ofTwo studies are reported that investigate how readily accessible and applicable ten force-dynamic categories are to novices in describing short episodes of human-technology interaction (Study 1) and that establish a measure of inter-coder reliability when re-classifying these episodes into force-dynamic categories (Study 2). The results of the first study show that people can easily and confidently relate their experiences with technology to the definitions of force-dynamic events (e.g. “The driver released the handbrake” as an example of restraint removal). The results of the second study show moderate agreement between four expert coders across all ten force-dynamic categories (Cohen’s kappa = .59) when re-classifying these episodes. Agreement values for single force-dynamic categories ranged between ‘fair’ and ‘almost perfect’, i.e. between kappa = .30 and .95. Agreement with the originally intended classifications of study 1 was higher than the pure inter-coder reliabilities. Single coders achieved an average kappa of .71, indicating substantial agreement. Using more than one coder increased kappas to almost perfect: up to .87 for four coders. A qualitative analysis of the predicted versus the observed number of category confusions revealed that about half of the category disagreement could be predicted from strong overlaps in the definitions of force-dynamic categories. From the quantitative and qualitative results, guidelines are derived to aid the better training of coders in order to increase inter-coder reliability.…
| Autor(en): | Jörn Hurtienne |
|---|---|
| URN: | urn:nbn:de:bvb:20-opus-194127 |
| Dokumentart: | Artikel / Aufsatz in einer Zeitschrift |
| Institute der Universität: | Fakultät für Humanwissenschaften (Philos., Psycho., Erziehungs- u. Gesell.-Wissensch.) / Institut Mensch - Computer - Medien |
| Sprache der Veröffentlichung: | Englisch |
| ISSN: | 2197-2796 |
| ISSN: | 2197-2788 |
| Erscheinungsjahr: | 2013 |
| Band / Jahrgang: | 1 |
| Heft / Ausgabe: | 1 |
| Seitenangabe: | 59-78 |
| Originalveröffentlichung / Quelle: | Yearbook of the German Cognitive Linguistics Association 2013, 1(1), 59-78. doi: https://doi.org/10.1515/gcla-2013-0005 |
| DOI: | https://doi.org/10.1515/gcla-2013-0005 |
| Allgemeine fachliche Zuordnung (DDC-Klassifikation): | 0 Informatik, Informationswissenschaft, allgemeine Werke / 00 Informatik, Wissen, Systeme / 004 Datenverarbeitung; Informatik |
| Freie Schlagwort(e): | force dynamics; humantechnology interaction; image schemas; inter-coder reliability |
| Datum der Freischaltung: | 26.11.2020 |
| Datum der Erstveröffentlichung: | 18.12.2013 |
| Anmerkungen: | This publication is with permission of the rights owner freely accessible due to an Alliance licence and a national licence (funded by the DFG, German Research Foundation) respectively. |
| Lizenz (Deutsch): |  Deutsches Urheberrecht |