LIVES – Swiss Centre of Expertise in Life Course Research: Methodological Research

LIVES-FORS Mixed Mode Experiment, 2012-2013

In 2012, I collaborated with Dominique Joye (UNIL) and Michèle Ernst Stähli (FORS), leading on the design, implementation and analysis of a large-scale (n=3,600) mixed mode experiment in the French-speaking region of Switzerland. The study was conducted in the context of a work package of the National Centre for Competence in Research “LIVES – Overcoming Vulnerability: Life Course Perspectives”, funded by the Swiss National Science Foundation (Individual Project 15 – Life transitions in context: Towards an integrated methodological framework for studying the impact of critical events, led by Dominique Joye). The experiment was conducted alongside Round 6 of the European Social Survey (2012), enabling comparisons between the standard ESS data based on face-to-face interviewing, and alternative single and mixed mode designs (telephone only, telephone plus mail; web plus mail; mail only).

The vehicle for the experiment was a survey called ‘Wellbeing and unease in Suisse romande’ (Bien-être et mal-être en Suisse romande) and included questions from the rotating module of the ESS on personal wellbeing, which are known to be highly sensitive to mode effects. Additional questions intended for methodological research purposes were included, plus other questions from the ESS core questionnaire (mainly socio-demographic measures). The data were used in several publications and a PhD project (also funded by LIVES) by Rosa Sanchez Tomé, supervised by me, with Dominique Joye as co-supervisor. The thesis investigated how mode of data collection affects the measurement of personal and subjective wellbeing in surveys.

Outputs

Technical report and data

  • Roberts, C., Joye, D., Ernst Stähli, M., and Sanchez Tome, R.(2016). Mixing modes of data collection in Swiss social surveys: Methodological Report of the LIVES-FORS Mixed Mode Experiment. LIVES Working Paper Series, 2016/ 48. University of Lausanne: NCCR LIVES.
  • Appendix to the Methodological Report, including fieldwork documents and questionnaires.
  • Roberts, C., Ernst Stähli, M., Joye, D., and Sanchez Tome, R. (2022). Well-being and Unease in French-speaking Switzerland. The LIVES-FORS Mixed Mode Experiment 2012 [Dataset]. FORS – Swiss Centre of Expertise in the Social Sciences, University of Lausanne. Distributed by SWISSUbase, Lausanne, 2022. DOI: https://doi.org/10.48573/88g0-bw17
  • Roberts, C., Ernst Stähli, M., Joye, D., and Sanchez Tome, R. (2022). Well-being and Unease in French-speaking Switzerland. The LIVES-FORS Mixed Mode Experiment 2012 (restricted access) [Dataset]. FORS – Swiss Centre of Expertise in the Social Sciences, University of Lausanne. Distributed by SWISSUbase, Lausanne, 2022. DOI: https://doi.org/10.48573/yx0e-qs55

Publications

  • Roberts, C. and Voorpostel, M. (2023) Combining data collection modes in longitudinal studies. In D. Spini and E. Widmer (Eds.) Withstanding Vulnerability throughout Adult Life. Dynamics of Stressors, Resources, and Reserves. Springer. Chapter 22, pp. 359-373.
  • Sanchez Tomé, R. (2018). The impact of mode of data collection on measures of subjective wellbeing. PhD Thesis, University of Lausanne.
  • Roberts, C., and Vandenplas, C. (2017). Estimating components of mean-squared error to evaluate the benefits of mixing data collection modes. Journal of Official Statistics, 33(2): 303-334. http://dx.doi.org/10.1515/JOS-2017-0016
  • Roberts, C., Vandenplas, C., and Ernst Staehli, M. (2014). Evaluating the impact of response enhancement methods on the risk of nonresponse bias and survey costs. Survey Research Methods, 8(2): 67-80. https://doi.org/10.18148/srm/2014.v8i2.54.
  • Roberts, C., Vandenplas, C., Ernst Stähli, M., and Joye, D. (In revision). Optimising mixed mode designs to improve the representativeness of survey samples. (Draft paper available on request.)

Presentations

  • Sanchez Tome, R., Roberts, C., and Joye, D. (2017). Mode effects in answers to sensitive open-ended questions. Paper presented at the 7th Conference of the European Survey Research Association, Lisbon, Portugal, 17th-21st July 2017.
  • Roberts, C., Ernst Stähli, M., Joye, D., and Sanchez Tome, R. (2015). Combining web and mail sequentially – error and cost advantages in the Swiss context.  Paper presented at 6th Conference of the European Survey Research Association, Reykjavik, Iceland, July 13-17, 2015.
  • Ernst Stähli, M., Roberts, C., Sanchez Tome, R. and Joye, D. (2015). Comparing estimates across survey designs – are mode effects the greatest cause for concern? Paper presented at 6th Conference of the European Survey Research Association, Reykjavik, Iceland, July 13-17, 2015.
  • Sanchez Tome, R., Roberts, C., Ernst Stähli, M., and Joye, D. (2015). Mode effects on measures of wellbeing: a comparison between uni- and multi-dimensional approaches. Paper presented at 6th Conference of the European Survey Research Association, Reykjavik, Iceland, July 13-17, 2015.
  • Roberts, C., Vandenplas, C., Joye, D., Ernst Stähli, M., Sanchez Tome, R., and Allum, N. (2015). Estimating components of Mean-Squared Error as a means to evaluate mixed mode solutions to noncoverage error in telephone surveys. 2015 International Total Survey Error Conference, Baltimore, MD., USA – 19-22/9/2015
  • Ernst Stähli, M., Joye, D., and Roberts, C. (2015). Alternatives au problème de couverture dans les enquêtes CATI : Enseignements de l’expérience mixed-mode de LIVES-FORS. Paper presented at the Workshop FORS-BFS ‘Erfahrungen mit mixed mode in Surveys’, Neuchâtel, Switzerland, March 16, 2015.
  • Sanchez Tome, R., Roberts, C., Ernst Staehli, M., and Joye, D. (2014). Mode effects in measures of subjective well-being – Implications for cross-survey comparisons. Paper presented at the 67th Annual Conference of the World Association for Public Opinion Research, Nice, France – 4-6/9/2014
  • Roberts, C., Vandenplas, C., Ernst Staehli, M., Joye, D., and Sanchez Tome, R. (2014). A comparison of the impact of single and mixed mode follow-ups of nonrespondents on survey errors and costs. Paper presented at the VI European Congress of Methodology, Utrecht, Netherlands – 23-25/7/2014
  • Ernst Stähli, M., Roberts, C., and Joye, D. (2013). Coverage, representativity and data quality in mixed mode surveys: Some results from a Swiss experiment. Paper presented at the 6th Webdatanet meeting and conference ‘Mixed mode and multi-method research’, Reykjavik, Iceland, 16/9/2013
  • Roberts, C., Ernst Staehli, M., and Joye, D. (2013). Mixed mode solutions to coverage and nonresponse error: Evaluation of the cost-error trade-off. Paper presented at the 5th Conference of the European Survey Research Association, Ljubljana, Slovenia – 15-19/7/2013

Methodological Challenges in Measuring Vulnerability

As part of my role as a survey methodologist in the NCCR LIVES, I was involved in advising on methodological aspects of the design of various surveys conducted in the context of the Individual Projects (IPs). This included contributions to the design of the respondent-driven sampling strategy for the LIVES-FORS Cohort Study, the design of the LIVES Event History Calendar used in the Cohort Study and the Swiss Household Panel survey, and a later study investigating mode preferences among participants in the Cohort Study, to inform data collection strategies in subsequent waves. Together with Michel Oris, Dominique Joye and Michèle Ernst Stähli, I collaborated closely with other LIVES researchers on a broader reflection around the methodological and substantive challenges involved in measuring vulnerability across the life course using surveys – particularly among hard-to-reach, hard-to-survey populations. This reflection took the form of a one-day international workshop organised by me in 2012, and a co-edited book on the challenges of ‘surveying vulnerabilities’.

Outputs

  • Oris, M., Roberts, C., Joye, D., and Ernst Stähli, M. (2016). Surveying human vulnerabilities across the Life Course: Balancing substantive and methodological challenges. In M. Oris, C. Roberts, D. Joye, and M. Ernst Stähli. (Eds.) Surveying human vulnerabilities across the Life Course.  Springer ‘Life Course Research and Social Policies’ series. Chapter 1, pp. 1-25.
  • Oris, M., Roberts, C., Joye, D., and Ernst Stähli, M. (Eds.) (2016). Surveying human vulnerabilities across the Life Course.  Springer ‘Life Course Research and Social Policies’ series.
  • Morselli, Davide; Spini, Dario; Le Goff, Jean-Marie; Gauthier, Jacques-Antoine; Brändle, Karen; Mugnari, Estelle; Dasoki, Nora; Roberts, Caroline; Bernardi, Laura; Bühlmann, Felix; Ryser, Valérie-Anne; Tillmann, Robin; Kühr, Judith & Bumbaru, Alexandra. (2013). Assessing the performance of the Swiss Panel LIVES Calendar: Evidence from a pilot study. LIVES Working Paper, 2013/28
  • Johnson, S. (2020). Investigating the impact of switching to web in a longitudinal telephone survey: potential effects on sample composition and attrition. Master’s dissertation, defended January 2020 – University of Lausanne. Supervisor: C. Roberts.
  • Spini, D., et al. (2019). The LIVES-FORS Cohort Survey: A longitudinal, diversified sample of young adults who have grown up in Switzerland. Longitudinal and Life Course Studies, 10(3): 399-410.

Scoping Study for a Swiss Time Use Survey 2023

In 2023, I received LIVES Seed Money to fund a scoping study to assess the demand for and feasibility of conducting a Time Use Survey in Switzerland using digital data collection methods. The feasibility evaluation followed a ‘respondent-centred’ survey design approach, focused on identifying the main barriers to participation in an app-based time-use study (with a particular focus on the relative weight of perceived response burden, digital mistrust and data privacy concerns) and possible ways of addressing them. At the end of 2023, I was awarded additional LIVES Seed Money, and research support funding from the Faculty of Social and Political Sciences, to extend the feasibility assessment in 2024 with a pilot survey to test an app-based diary tool for respondents to record their activities.

Please check back soon for more information!

Collaborators: Daniel Oesch (UNIL), Marieke Voorpostel (FORS), and Oliver Lipps (FORS)