eprintid: 10199776
rev_number: 12
eprint_status: archive
userid: 699
dir: disk0/10/19/97/76
datestamp: 2024-11-08 16:05:31
lastmod: 2024-11-08 16:05:31
status_changed: 2024-11-08 16:05:31
type: proceedings_section
metadata_visibility: show
sword_depositor: 699
creators_name: Alshahwan, Nadia
creators_name: Blasi, Arianna
creators_name: Bojarczuk, Kinga
creators_name: Ciancone, Andrea
creators_name: Gucevska, Natalija
creators_name: Harman, Mark
creators_name: Krolikowski, Michal
creators_name: Rojas, Rubmary
creators_name: Martac, Dragos
creators_name: Schellaert, Simon
creators_name: Ustiuzhanina, Kate
creators_name: Harper, Inna
creators_name: Jia, Yue
creators_name: Lewis, Will
title: Enhancing Testing at Meta with Rich-State Simulated Populations
ispublished: pub
divisions: UCL
divisions: B04
divisions: F48
keywords: Software Testing, Cyber Cyber Digital Twins, Simulation-Based
Testing, Machine Learning
note: This version is the author accepted manuscript. For information on re-use, please refer to the publisher’s terms and conditions.
abstract: This paper reports the results of the deployment of Rich-State Simulated Populations at Meta for both automated and manual testing. We use simulated users (aka test users) to mimic user interactions and acquire state in much the same way that real user accounts acquire state. For automated testing, we present empirical results from deployment on the Facebook, Messenger, and Instagram apps for iOS and Android Platforms. These apps consist of tens of millions of lines of code, communicating with hundreds of millions of lines of backend code, and are used by over 2 billion people every day. Our results reveal that rich state increases average code coverage by 38%, and endpoint coverage by 61%. More importantly, it also yields an average increase of 115% in the faults found by automated testing. The rich-state test user populations are also deployed in a (continually evolving) Test Universe; a web-enabled simulation platform for privacy-safe manual testing, which has been used by over 21,000 Meta engineers since its deployment in November 2022.
date: 2024-05-31
date_type: published
publisher: ACM
official_url: https://doi.org/10.1145/3639477.3639729
oa_status: green
full_text_type: pub
language: eng
primo: open
primo_central: open_green
verified: verified_manual
elements_id: 2284859
doi: 10.1145/3639477.3639729
lyricists_name: Harman, Mark
lyricists_id: MHARM36
actors_name: Harman, Mark
actors_id: MHARM36
actors_role: owner
full_text_status: public
pres_type: paper
publication: Proceedings of the 46th International Conference on Software Engineering: Software Engineering in Practice
pagerange: 1-12
pages: 12
event_title: ICSE-SEIP '24: 46th International Conference on Software Engineering: Software Engineering in Practice
event_location: PORTUGAL, Lisbon
event_dates: 14 Apr 2024 - 20 Apr 2024
book_title: Proceedings of the 46th International Conference on Software Engineering: Software Engineering in Practice
citation:        Alshahwan, Nadia;    Blasi, Arianna;    Bojarczuk, Kinga;    Ciancone, Andrea;    Gucevska, Natalija;    Harman, Mark;    Krolikowski, Michal;                             ... Lewis, Will; + view all <#>        Alshahwan, Nadia;  Blasi, Arianna;  Bojarczuk, Kinga;  Ciancone, Andrea;  Gucevska, Natalija;  Harman, Mark;  Krolikowski, Michal;  Rojas, Rubmary;  Martac, Dragos;  Schellaert, Simon;  Ustiuzhanina, Kate;  Harper, Inna;  Jia, Yue;  Lewis, Will;   - view fewer <#>    (2024)    Enhancing Testing at Meta with Rich-State Simulated Populations.                     In:  Proceedings of the 46th International Conference on Software Engineering: Software Engineering in Practice.  (pp. pp. 1-12).  ACM       Green open access   
 
document_url: https://discovery.ucl.ac.uk/id/eprint/10199776/8/Harman_RPS4.pdf