Methodology of Training AI Agents To Evaluate Videos with Imitation of Human Evaluation: A Sociological Aspect

This article presents a sociologically grounded methodology for training AI agents to rate videos, simulating human ratings while taking into account socially conditioned models of content perception by real users. It should be noted that there are practically no articles directly on the methodology of training AI agents to simulate human evaluation of videos with sociological evaluation in journals for 2021–2026, but there are similar publications (in foreign and Russian sources), the authors of which focus on AI assessment of social situations in video games and sociological simulation of behaviour. The study addresses the discrepancy between algorithmic ratings and video perception by different social groups, which arises due to the algorithms’ reliance on formalized metrics and their ignorance of the sociocultural characteristics of viewer perception. The methodology draws on theoretical approaches from symbolic interactionism, social representation theory, and social constructivism, as well as concepts from digital sociology and media perception theory. It relies on four key principles: social representativeness of data, modeling of social processes, accounting for variability in perception, and transparency of artificial intelligence (AI) decisions. A system of sociological criteria for assessing the humanlikeness of AI decisions has been developed. Mechanisms for validating results are proposed, including calculating the agreement coefficient between AI and social group assessments, determining the proportion of AI decisions that users cannot distinguish from human ones, analyzing the percentage reduction in appeals compared to traditional systems, and assessing the cultural adaptability index, namely, the model’s ability to operate correctly in different sociocultural contexts. This methodology will bridge the gap between algorithmic and social assessment of video content, and its implementation will increase the relevance of AI systems by taking into account group differences in viewer perception, thereby contributing to the creation of more balanced and socially appropriate decisions in the digital media space. In the long term, this will allow for more sustainable content evaluation practices that focus not only on formal criteria, but also on the dynamics of social norms and audience values.

1. Argyle L.P., Busby E.C., Fulda N., Gubler J.R., Rytting C., Wingate D. (2023) Out of One, Many: Using Language Models to Simulate Human Samples. Political Analysis. Vol. 31. Is. 3. P. 337–351. DOI: 10.1017/pan.2023.2

2. Bandura A. (2000) Social Learning Theory. St. Petersburg: Yevraziya.

3. Berger P., Luckmann T. (1995) The Social Construction of Reality: A Treatise in the Sociology of Knowledge. Moscow: Medium.

4. Blumer G. (2017) Symbolic Interactionism: Perspective and Method. Moscow: Elementarnye formy.

5. Bourdieu P. (2005) La Distinction: Critique sociale du jugement. Ekonomicheskaya sotsiologiya. Vol. 6. No. 3. P. 25–48.

6. Cohen J.A (1960) Coefficient of Agreement for Nominal Scales. Educational and Psychological Measurement. 1960. Vol. 20. Is. 1. P. 37–46. DOI: 10.1177/001316446002000104

7. Fleiss J.L. (1971) Measuring Nominal Scale Agreement among Many Raters. Psychological Bulletin. Vol. 76. Is. 5. P. 378–382. DOI: 10.1037/h0031619

8. Kardanova Ye.Yu. (ed.) (2025) Novyye podkhody k otsenivaniyu: iskusstvennyy intellekt kak drayver izmeneniy v obrazovanii [New approaches to assessment: Artificial Intelligence as a driver of change in education]. Moscow: NIU VSHE.

9. Katz E. (1987) Communications Research since Lazarsfeld. Public Opinion Quarterly. Vol. 51. Special Issue. P. S25–S45.

10. Krishnan N. (2025) AI Agents: Evolution, Architecture, and Real-World Applications. arXiv Preprint. DOI: 10.48550/arXiv.2503.12687

11. Lazarsfeld P., Berelson B., Gaudet H. (1944) The People’s Choice: How the Voter Makes Up His Mind in a Presidential Campaign. Princeton: Princeton University Press.

12. Lippmann W. (2004) Public Opinion. Moscow: Institut Fonda “Obshchestvennoye mneniye”.

13. Lupton D. (2017) Digital Sociology. London: Routledge.

14. Mitchell W.J.T. (1994) Picture Theory: Essays on Verbal and Visual Representation. Chicago: University of Chicago Press.

15. Moscovici S. (1995) Social Representations: A Historical Perspective. Psikhologicheskiy zhurnal. Vol. 16. No. 1. P. 3–18.

16. Puzanova Zh.V., Kozhoridze G.G., Kozhoridze D.G. (2025) Generative AI and Sociology: Analyzing Virtual Respondent Technology. Sotsiologiya: metodologiya, metody, matematicheskoye modelirovaniye. No. 60. P. 216–246. DOI: 10.19181/4m.2025.34.1.6

17. Qu X., Damoah A., Sherwood J., Liu P., Jin Ch., Chen L., Shen M., Aleisa N., Hou Z., Zhang Ch., Gao L., Li Y., Yang Qu., Wang Qu., De Souza Ch. (2025) A Comprehensive Review of AI Agents: Transforming Possibilities in Technology and Beyond. arXiv Preprint. DOI: 10.48550/arXiv.2508.11957

18. Safonova Yu.A., Subocheva O.N., Korshkova A. S. (2023) Agency of Artificial Autonomous Systems as a Factor in the Transformation of Society. Sotsiologiya. No. 6. P. 116–122.

19. Santavirta S., Wu Y., Suominen L., Nummenmaa L. (2025) GPT-4V Shows Human-like Social Perceptual Capabilities at Phenomenological and Neural Levels. Imaging Neuroscience. Vol. 3. DOI: 10.1162/IMAG.a.134

20. Silverstone R. (2007) Media and Morality: The Rise of Mediated Public Conscience. Cambridge: Polity Press.

21. Spearman C. (1904) The Proof and Measurement of Association between Two Things. The American Journal of Psychology. Vol. 15. Is. 1. P. 72–101.