Reliability and validity of children’s advertising exposure measures

Suzanna Opree (Suzanna Opree is based at Erasmus School of History, Culture and Communication, Erasmus University Rotterdam, Rotterdam, The Netherlands)
Moniek Buijzen (Erasmus School of Social and Behavioral Sciences, Erasmus University Rotterdam, Rotterdam, The Netherlands)
Eva van Reijmersdal (Amsterdam School of Communication Research, University of Amsterdam, Amsterdam, The Netherlands)

Young Consumers

ISSN: 1747-3616

Article publication date: 8 August 2021

Issue publication date: 8 October 2021

2149

Abstract

Purpose

The aim of this study is to determine which of previously used survey measures can be considered the most appropriate to assess children’s advertising exposure. First, three levels of content specificity for assessing children’s exposure to advertising were distinguished as follows: exposure to the medium, exposure to broad content and exposure to specific (i.e. commercial) content. Second, using longitudinal data from 165 children between 8 and 11 years old, the test-retest reliability and content validity of survey measures from all three levels were examined.

Design/methodology/approach

Due to societal concerns about the effects of advertising on children’s well-being, research into this topic is expanding. To enhance knowledge accumulation and bring uniformity to the field, a validated standard survey measure of advertising exposure is needed. The aim of this study is to provide such measures for television and internet advertising.

Findings

The findings suggest that all measures provided solid estimates for children’s television and internet advertising exposure. Yet, due to minor differences in reliability and validity, it may be concluded that television advertising exposure can best be measured by asking children how often they watch certain popular (commercial) television networks, either weighting or not weighting for advertising density. Internet advertising exposure can best be measured by asking children how often they use the internet or how often they visit certain popular websites, weighting for advertising density.

Originality/value

The current measures for children’s advertising exposure through traditional media can easily be adapted to fit new media.

Keywords

Citation

Opree, S., Buijzen, M. and van Reijmersdal, E. (2021), "Reliability and validity of children’s advertising exposure measures", Young Consumers, Vol. 22 No. 4, pp. 660-674. https://doi.org/10.1108/YC-10-2020-1243

Publisher

:

Emerald Publishing Limited

Copyright © 2021, Suzanna Opree, Moniek Buijzen and Eva van Reijmersdal.

License

Published by Emerald Publishing Limited. This article is published under the Creative Commons Attribution (CC BY 4.0) licence. Anyone may reproduce, distribute, translate and create derivative works of this article (for both commercial & non-commercial purposes), subject to full attribution to the original publication and authors. The full terms of this licence may be seen at http://creativecommons.org/licences/by/4.0/legalcode.


Introduction

Western children grow up in a heavily commercialized media environment. Expenditures on children’s advertising have been increasing for decades, as have concerns about advertising’s potential side effects (Oates et al., 2014). Children are believed to be particularly vulnerable to the effects of advertising, and parents, psychologists and policymakers are concerned that advertising may lead to decreased happiness, higher weight status and negative self-perceptions (Kasser and Linn, 2016). Because of the growing concerns about the effects of advertising on children’s well-being, research related to this topic is becoming increasingly important.

The most appropriate method for studying the effects of advertising depends on the outcome variable of interest. For example, experiments can be used to investigate the immediate effects of advertising exposure on children’s product desires and product purchase requests. However, the presumed long-term effects of advertising, such as increased parent-child conflict and materialism, are likely the result of cumulative exposure and can, therefore, best be investigated with longitudinal surveys or diary studies (Potter, 2014). Thus, far, only a handful of scholars have conducted longitudinal research on the effects of advertising among children, and these authors have primarily conducted survey studies (Opree et al., 2012; Opree et al., 2014; Opree et al., 2016). In previous survey research, scholars have used a variety of scales to assess children’s advertising exposure, making it difficult to compare their findings. To enhance knowledge accumulation and bring uniformity to the field, a validated standard survey measure of advertising exposure is needed. Therefore, the aim of this study is to determine which of the previously used survey measures can be considered the most appropriate to assess children’s advertising exposure. This study focuses on advertising exposure measurements related to television and the internet because these media are most frequently used by advertisers to reach a young target audience (Richins, 2017).

Some might argue that the most precise survey measure of children’s advertising exposure is their exposure to specific commercials or advertisements. For instance, Buijzen and Valkenburg (2003b, 2005) measured children’s advertising exposure by presenting them with the titles of 10 commercials and asking how often they had seen these commercials. However, this approach has three important shortcomings. First, this approach cannot be used in longitudinal research because the commercials or advertisements that are featured during the first measurement occasion may no longer be featured at the second measurement occasion. The second limitation of this approach is that it involves recall – something many children will experience problems with because their cognitive capacities are yet to fully mature (Borgers, 2003). Third, in the advertising literature, the memory of advertisements is in fact often considered as a cognitive effect of exposure and a first step in the persuasion process (Du Plessis, 1994). It should, therefore, be disentangled from advertising exposure itself. Hence, following Slater’s (2004, p. 168) definition of exposure (“the extent to which audience members have encountered specific messages or classes of messages/media content”), we define advertising exposure as encountering – but not necessarily processing – commercials or advertisements.

In previous studies on exposure to news (Wonneberger, 2011), health messages (Romantan et al., 2008), substance use in motion pictures (Sargent et al., 2008) and sex content (Annenberg Media Exposure Research Group, 2008), exposure has been measured at various levels of content specificity (Romantan et al., 2008; Slater, 2004). Content specificity can be thought of as a continuum, with general measures related to media exposure at the low end of the spectrum, measures related to broad content in the middle and measures related to specific content or combining data sources (e.g. enriching exposure data with advertising density or content analysis data, Slater, 2004) at the high end (Romantan et al., 2008). In the present study, the quality of advertising exposure measures from the various levels of content specificity will be assessed by determining their stability and content validity. Though the test-retest reliability and content validity have not been previously tested for most of the measurements included in this study, several scholars suggest that the predictive value of measures increases with their content specificity (Romantan et al., 2008; Slater, 2004).

To construct television and internet advertising exposure measures on all three levels of content specificity, survey data on children’s media exposure and industry data on advertising expenditures needed to be matched. Also, to construct appropriate measurements, both data sets had to have been collected simultaneously – limiting the available options. This manuscript is based on two existing data sets that the authors collected in 2011. In this data set, television exposure refers to synchronous broadcast television viewing and internet exposure to children’s website use. Since 2011 the media landscape has changed and television exposure can now also be operationalized as on-demand asynchronous broadcast television viewing on/or viewing the content of streaming services such as Netflix. Furthermore, internet users can now also be operationalized as the use of all sorts of applications on smart devices (e.g. social media apps such as Instagram, Snapchat and TikTok, video-chatting apps and gaming apps). Still, recent time use research indicates that children predominantly view TV on a TV set rather than on other devices and that most viewing on a TV set is live rather than time shifted. Also, on average, children still browse websites for an hour per day (Rideout and Robb, 2019). This shows that the operationalizations of the measures that were included in this study are still relevant today.

Content specificity continuum

Below, the different levels of content specificity of measuring children’s advertising exposure are explained. For each of the measures along with the continuum, references to articles using that approach to measure children’s advertising exposure in survey research are provided. The measures that were used in the studies below formed the direct inspiration for the measures that were included in the current study. To avoid repetition throughout this manuscript, more detailed information on these measures is provided in the method section.

Level 1: media exposure.

Measures at the low end of the content specificity continuum address exposure to the medium, which can be measured in multiple ways. The most popular method is to ask respondents to indicate how many hours and minutes they spend using a medium on an average day (Jordan et al., 2007). However, this is a difficult question even for adults because respondents must first decide what an “average” day is for them and then estimate the time they spend using a medium on such an average day. Although adding a reference period might simplify the question (e.g. asking how many hours and minutes a respondent has used a medium the previous day or week), the question remains difficult. Even adults seem unable to provide accurate estimates of their media time (Araujo et al., 2017; Greenberg et al., 2005). Because of their limited cognitive abilities (de Leeuw et al., 2004), this task is likely to result in even more unreliable estimates among children.

A frequently used and better alternate is to measure children’s media exposure using Likert scales (Jordan et al., 2007). Respondents are asked to indicate how often they use a certain medium and are offered a set of response categories, such as “never,” “sometimes,” “often” and “very often.” Although this is a relative measure using vague quantifiers and respondents might interpret the categories in different ways, it provides information about an individual’s sense of his or her general media use compared to that of others. Previous research has indicated that vague quantifiers may be a more “useful measurement tool for behavioral data” compared to numeric response options due to their higher predictive validity (Al Baghal, 2014). Yet, to increase preciseness, researchers may choose to ask the question of how often respondents use a certain medium separately for mornings, afternoons and evenings (Van der Voort and Vooijs, 1990; Vooijs et al., 1987). Exposure to television has been frequently used as a proxy for television advertising exposure in the early youth and advertising studies of the 1970s (Atkin, 1975a, 1975b; Churchill and Moschis, 1979; Moschis and Churchill, 1978; Ward and Wackman, 1971) and the early 2000s (Pine and Nash, 2002).

Level 2: broad content.

In the middle of the specificity, continuum are measures of broad media content (Romantan et al., 2008). Asking children about the content they have consumed may lead to a more accurate estimation of their actual media use (Slater, 2004), and hence, their advertising exposure. With regard to television use, researchers can ask children about the network channels and programs they have watched and for the internet, they can ask about the websites children have visited (Greenberg et al., 2005; Vandewater and Lee, 2009). Generally, exposure to content can be measured using the following two different strategies: free recall or aided recall. With free recall, respondents are asked to write down all the content they have consumed in a certain period of time. With aided recall, respondents are provided with a list and asked to indicate which content they have consumed (Du Plessis, 1994). Although free recall may work for adults, this task is cognitively too demanding for children (Borgers, 2003). Jordan et al. (2007) demonstrated in a focus group study about television use that even adolescents were unable to recall watching certain programs unless they were prompted with a list of programs. Exposure to broad television content has been used as a proxy for children’s television advertising exposure by Rozendaal et al. (2009).

Level 3: specific content.

At the higher end of the specificity, continuum are measures of specific content, which can also be derived by combining sources. The third level on the specificity continuum involves exposure to commercial media content only. Media effects are less a result of how much media content people consume than how much of a certain type of media content they consume (Slater, 2004), and children’s exposure to commercial media is traditionally believed to be a better predictor of the effects of advertising than children’s consumption of general media (Kunkel, 2001). Exposure to commercial television content has been used as a proxy for television advertising exposure by several scholars in the field (Buijzen and Valkenburg, 2000; Opree et al., 2012; Opree et al., 2014; Opree et al., 2016; Pine and Nash, 2002).

Slater (2004) reasoned that the predictive power of advertising exposure measures can further be improved by enriching survey data with advertising data available from Nielsen and other market research firms. For television advertising, these data contain an overview of all the commercials that were broadcast during a certain period of time. For each commercial, the day, time and network it was broadcast is registered, as well as its price. For internet advertising, the data contain an overview of the number of advertisements that was visible on a website during a certain period of time and the total advertising income for that website during that period. The data includes both advertisements on the website’s main page and its back pages (i.e. for Google that would be the pages with the search results). So far, in youth research, enriching survey data with advertising data has only been in studies investigating the impact of food and alcohol advertising (Buijzen, Bomhof and Schuurman, 2008; Buijzen, Schuurman and Bomhof, 2008; Grenard et al., 2013).

Method

Sample and procedure

The survey data set contained longitudinal data collected from 165 children 8–11 years of age (51.3% girls; mean age 9.94 years). The advertising literacy skills of 8–11-year-olds are yet to fully mature (Rozendaal et al., 2010), making them extra vulnerable to the effects of advertising and a particularly interesting group for researchers to study. The respondents were recruited through five primary schools located in different parts of The Netherlands – including schools in urban and rural areas and in more and less affluent areas to foster representativeness. The survey was administered twice, with a two-week interval in between, to all children in the 7th-grade (comparable to US 5th-grade level). All children were informed that the questionnaire addressed television use, internet use and general happiness and that they could withdraw from the study at any time. Prior to the implementation of the study, informed consent was obtained from both children and parents. The study was granted Institutional Review Board approval by the university’s ethical committee.

The questionnaire was pretested among children from the same age range as the intended research population. All measures were based on existing measures but were sometimes altered in consideration of the general guidelines for conducting research with children as follows: items were phrased as questions rather than statements (Buijzen and Valkenburg, 2003b), ambiguity and vagueness were avoided as much as possible (de Leeuw, 2003; de Leeuw et al., 2004) and for the closed-ended questions, four response options were provided because this has been shown to be the optimal number for children in this age group (Borgers, 2003).

Measures

Table 1 provides an overview of the advertising exposure measures that were used in this study. In this table, the measures are sorted by medium type (i.e. television vs internet advertising exposure, denoted by 1.x and 2.x) and ascending level of specificity (i.e. Levels 1 through 3, denoted by x.1, x.2 and x.3). Each item of each measure had four response categories, namely, 1 (never), 2 (sometimes), 3 (often) and 4 (very often). For each measure, a composite score was computed, averaging the scores on the separate items.

Television advertising exposure.

Level 1: media exposure (1.1). Exposure to the medium was measured with three items, combining the strategies of Pine and Nash (2002) to measure advertising exposure with exposure to the medium and of Van der Voort and Vooijs (1990) and Vooijs et al. (1987) to measure exposure for different parts of the day as follows:

  • How often do you watch television in the morning?

  • How often do you watch television in the afternoon? and

  • How often do you watch television in the evening? (Nwave1 = 159, M = 2.40, SD = 0.57; Nwave2 = 163, M = 2.47, SD = 0.57; rwave1−wave2 = 0.71 with p < 0.001).

Level 2: broad content (1.2). Exposure to broad content was measured as exposure to network channels (1.2a) and exposure to television programs (1.2b). Following the strategy of Rozendaal et al. (2009), content from both publicly funded and commercial networks were included. Though publicly funded networks in The Netherlands also broadcast advertising, the frequency of commercial blocks is considerably lower than for commercial networks. Exposure to networks was measured with six items. For each item, the children were asked to indicate how often they watched a certain network. The six networks that were most popular among 8–12-year-olds were included in the items: the publicly funded networks Nederland 1 and Nederland 3 (Z@pp) and the commercial networks RTL4, SBS6, Disney XD and Nickelodeon (Sikkema, 2009, 2012) (Nwave1 = 158, M = 2.28, SD = 0.44; Nwave2 = 163, M = 2.24, SD = 0.46; rwave1−wave2 = 0.74 with p < 0.001).

Exposure to television programs was measured with 12 items. For each item, the children were asked to indicate how often they watched a certain program. The programs were selected using data from the Dutch Audience Research Foundation. First, it was determined which programs on the most popular networks had the highest viewing ratings in the weeks leading up to the study. These were the children’s TV series Spangas, Klokhuis, iCarly, Ned’s Survival Guide, Phineas & Ferb and The Suite Life of Zack & Cody, the Dutch family programs Goede Tijden Slechte Tijden, Holland’s Got Talent and My Name Is, both the regular news program (Journal) and the news tailored to children (Jeugdjournaal) and the news-entertainment program Hart van Nederland. Because of changes in the broadcasting schedule ending two of the programs listed in the first wave, in the second wave Holland’s Got Talent was replaced with The Voice of Holland, and My Name Is with Ik Hou van Holland (Nwave1 = 159, M = 2.26, SD = 0.49; Nwave2 = 163, M = 2.47, SD = 0.52; rwave1−wave2 = 0.85 with p < 0.001).

Level 3: specific content (1.3). Exposure to commercial media content was also measured in the following four different ways: first, exposure to commercial networks (1.3a) and exposure to programs broadcast by commercial networks (1.3b). For both measures, a selection of items from the Level 2 measures was used. For exposure to commercial networks, only the items related to the commercial networks were included: RTL4, SBS6, Disney XD and Nickelodeon (Nwave1 = 158, M = 2.46, SD = 0.55; Nwave2 = 163, M = 2.38, SD = 0.61; rwave1−wave2 = 0.77 with p < 0.001). For exposure to programs broadcast by commercial networks, only items related to programs broadcast by commercial networks were selected, which were all the programs listed above except Spangas, Klokhuis, Journaal and Jeugdjournaal. Similar measures have been used by Buijzen and Valkenburg (2000), Opree et al. (2012), Opree et al. (2014), Opree et al. (2016) and Pine and Nash (2002) (Nwave1 = 159, M = 2.25, SD = 0.60; Nwave2 = 163, M = 2.25, SD = 0.61; rwave1−wave2 = 0.83 with p < 0.001).

Exposure to media content weighted for advertising density was measured in the following two different ways too: weighted exposure to network channels (1.3c) and weighted exposure to television programs (1.3d). In both cases, the items that were included in the Level 2 measure were weighted for (i.e. multiplied by) advertising density. This strategy has previously been used to measure exposure to food and alcohol advertising (Buijzen, Bomhof and Schuurman, 2008; Grenard et al., 2013). Advertising density on the networks and programs was derived from a data set provided by Nielsen market research. This confidential data set contained a list of commercials (including date, time, costs in Euros) aired on Dutch television.

Exposure to networks weighted for advertising density was measured with six items, each referring to a different network. For each item, children’s scores were weighted for the number of commercials aired by that channel (Nwave1 = 158, M = 28.28, SD = 5.72; Nwave2 = 163, M = 27.68, SD = 6.07; rwave1−wave2 = 0.75 with p < 0.001). Exposure to programs weighted for advertising density was measured with 12 items, each referring to a different program. For each item, children’s scores were weighted for the number of commercials aired during a typical episode of that program. To this end, for each program, the airing schedule was determined for the same three predetermined subsequent days (two weekdays and one weekend day). Then, the number of advertisements aired between 10 min prior to and 10 min after each broadcast was determined. Finally, this total number was divided by the number of broadcasts (Nwave1 = 159, M = 63.95, SD = 17.80; Nwave2 = 163, M = 56.31, SD = 14.12; rwave1−wave2 = 0.72 with p < 0.001).

Internet advertising exposure.

Level 1: media exposure (2.1). Exposure to the medium was measured with three items as follows: How often do you use the internet in the morning? How often do you use the internet in the afternoon? and How often do you use the internet in the evening? (Nwave1 = 159, M = 2.04, SD = 0.61; Nwave2 = 161, M = 2.10, SD = 0.62; rwave1−wave2 = 0.68 with p < 0.001).

Level 2: broad content (2.2). Exposure to broad content was measured with five items. For each item, the children were asked to indicate how often they visited a certain website. First, the ten most popular websites among 8–12-year-olds were determined. These were the search engine Google.nl, the video site Youtube.com, the children’s TV network sites Nickelodeon.nl and DisneyXD.nl, the gaming sites Speeleiland.nl, Spelletjes.nl, Spele.nl and Habbo.nl, the social network site Hyves.nl and the auction site Marktplaats.nl (Sikkema, 2009, 2012). Second, to control for advertising density in other measures, only websites for which Nielsen provided data on the number of advertisements were selected. Data on advertising density was available for only five of the 10 websites, namely, for Nickelodeon.nl, Speeleiland.nl, Spelletjes.nl, Spele.nl and Marktplaats.nl. These websites were included in the measure (Nwave1 = 159, M = 1.68, SD = 0.50; Nwave2 = 163, M = 1.80, SD = 0.52; rwave1−wave2 = 0.68 with p < 0.001).

Level 3: specific content (2.3). Exposure to specific content was first measured with a selection of websites from the Level 2 measure, namely, the three with the most advertising density: Spelletjes.nl, Spele.nl and Marktplaats.nl. The amount of advertising density per website was derived from confidential data provided by Nielsen market research. These data contained information about the advertising density for each of the websites that were included in this measure. Though all the websites from the Level 2 measure were commercial to some degree, we selected only three websites for the Level 3 measure to prevent the two measures from becoming too similar (Nwave1 = 159, M = 1.93, SD = 0.53; Nwave2 = 163, M = 1.88, SD = 0.57; rwave1−wave2 = 0.62 with p < 0.001). Second, exposure to media content weighted for advertising density was measured with five items. The same websites that were included in the Level 2 measure were used, weighing each item score for the mean number of advertisements displayed on the various pages of that website (Nwave1 = 158, M = 88.21, SD = 24.76; Nwave2 = 162, M = 85.19, SD = 25.64; rwave1−wave2 = 0.66 with p < 0.001).

Data analysis

As indicated in the introduction of this manuscript, we aim to determine the quality of advertising measures from the various levels of content specificity through their stability and content validity. Stability is measured with the test-retest reliability of measure: inter-individual scores need to be relative constant from one occasion to another (DeVellis, 2003; Noar, 2003). Above, the test-retest correlations for each of the measures were provided. Each of the measures yielded a correlation coefficient large in size and significant at p < 0.001, implying high test-retest reliability for all measures. Still, to determine whether certain measures resulted in more optimal test-retest reliability than others we have conducted one-on-one comparisons which are presented in the results section. Significant differences in test-retest reliability were assessed using the z-based Pearson-Filon (ZPF) statistic. This method is recommended by Raghunathan et al. (1996) for testing the significance for the difference between two correlations based on dependent groups (e.g. the same sample) but a different set of variables (e.g. the Wave 1 and Wave 2 scores for the Level 1 television advertising exposure measure vs the Wave 1 and Wave 2 scores for the Level 2 television advertising exposure measure). The ZPF statistics were obtained with the Cocor package in R (Diedenhofen, 2012).

Content validity involves item-sampling adequacy: does a scale measure what it intends to measure (DeVellis, 2003; Noar, 2003)? It can be assessed through criterion validity and construct validity. Criterion validity involves the relationship of the scale to other scales that presumably measure the same construct. In this case, it may be expected that measures of the same type of advertising exposure will be correlated across the three levels. Construct validity involves the relationship of the scale with established measures of other constructs: does the scale act like the variable it is supposed to measure and does it relate to other constructs in expected ways (DeVellis, 2003; Noar, 2003)? Here, a distinction is made between convergent validity and discriminant validity. With convergent validity a scale should relate to other constructs it is supposed to be related to. In this study, convergent validity will be assessed by studying the correlations between the advertising exposure measures and advertised product desire and materialism. Children’s exposure to advertising is expected to be positively related to advertised product desire and materialism. The more children are exposed to advertising, the more they long for heavily advertised products (Buijzen and Valkenburg, 2003b; Rozendaal et al., 2009; Opree et al., 2014) and the more materialistic they become in general (Buijzen and Valkenburg, 2003a, 2003b; Opree et al., 2014).

To measure advertised product desire, children were presented with a list of six products that appeal to girls and boys and to younger and older children (i.e. foods, toys, computer games, DVDs, magazines and shoes) (based on Buijzen and Valkenburg, 2003b). The Nielsen market research data showed that these were the most frequently advertised product types during children’s programming. For each of the products, the children were asked how often they wanted to own the product when they saw it in an advertisement (Opree et al., 2011, 2014; Rozendaal et al., 2009). The response options for the questions ranged from 1 (almost never) to 4 (very often). The item scores were averaged to create scale scores (Wave 1: α = 0.72, M = 1.76, SD = 0.50; Wave 2: α = 0.74, M = 1.71, SD = 0.51). Materialism was measured with the short version of the material values scale for children (MVS-c; developed and validated by Opree et al., 2011). The answer categories ranged from 1 (no, not at all) to 4 (yes, very much), with higher scores reflecting higher levels of materialism. Item scores were averaged to create scale scores (Wave 1: α = 0.70, M = 1.77, SD = 0.43; Wave 2: α = 0.79, M = 1.61, SD = 0.46).

With discriminant validity, a scale should be unrelated to other constructs it is supposed to be unrelated too. In this study, discriminant validity will be assessed by studying the correlations between the advertising exposure measures and gender and age. Children’s exposure to advertising is believed to be unrelated to gender and age because boys and girls and children within the age range of this sample (i.e. 8–11) share similar media consumption patterns, spending the same amount of time watching television or browsing websites (Rideout et al., 2010; Ofcom, 2017).

Results

Stability

For children’s television advertising exposure several significant differences in test-retest reliability were found. First, the media exposure measure (1.1) had lower test-retest reliability than the program measures (p < 0.001 for 1.2b and p < 0.01 for 1.3b). Second, the broad networks measure (1.2a) had lower test-retest reliability (p < 0.05) than the commercial program measure (1.3b) and, similarly, the commercial network (1.3a) measure had lower test-retest reliability (p < 0.05) than the broad program measure (1.2b). Interestingly, both the broad and commercial program measures also outperformed the measures in which exposure was weighted for advertising density: the difference in test-reliability between measure 1.2b and 1.3c was significant at p < 0.01, between 1.2b and 1.3d at p < 0.001, between 1.3b and 1.3c at p < 0.05 and between 1.3b and 1.3d at p < 0.001. In sum, the program measures seemed to have the most optimal test-retest reliability – this comes as no surprise given that only these measures had test-retest reliability above 0.80.

For internet advertising exposure no significant differences in test-retest reliability were found, with the exception of the measure pertaining to commercial websites (2.3a), which had lower test-retest reliability than the measures pertaining to the use of all websites and the measure in which this use was weighted for advertising density (p < 0.05 for both 2.2 and 2.3b). Hence, for capturing children’s internet advertising exposure, it seems best to use the latter measures and/or children’s general internet use.

Content validity

The content validity of the advertising exposure measures was assessed by means of Pearson correlation coefficients (Nwave1 = 159; Nwave2 = 163). Similar approaches for determining content validity were used in previous research (Opree et al., 2011; Rozendaal et al., 2016).

Criterion validity

Criterion validity was assessed using the correlation coefficients between the different advertising exposure measures. In Table 2, the correlations between the seven television advertising exposure measures are presented on top. The correlation coefficients for Wave 1 are presented first, and the correlation coefficients for Wave 2 second. All correlations were significant at p < 0.001, indicating high criterion validity. Table 2 also contains the correlations between the four internet advertising exposure measures. In both waves, the general internet use measure (2.1) yielded very low criterion validity. The correlations with the other measures were either small or nonsignificant. The correlations between the three other measures, however, were large in size and significant at p < 0.001.

Construct validity

The construct validity of the advertising exposure measures was assessed by means of their correlations with gender, age, advertised product desire and materialism. Advertising exposure was believed to be unrelated to gender and age but positively correlated to advertised product desire and materialism. The construct validity of the different measures can be derived from the number of correlations in the expected direction. In Table 3, the correlation coefficients are presented for both types of advertising exposure (i.e. television vs internet) and both waves separately.

With regard to the various television advertising exposure measures, the correlations with gender and age were not significant and the correlations with advertised product desire were positive at both waves. The findings related to materialism were mixed. The broad program measure was not associated with materialism at either wave, and the commercial program measure (1.3b) and program weighted measure (1.3d) were associated with materialism at Wave 1 but not at Wave 2. Hence, they scored slightly lower on construct validity than the other measures. For internet advertising exposure, the correlations with gender and age were also not significant, with the exception of the measure pertaining to the use of all websites (2.2) which was negatively related to age at Wave 1. All the measures were positively related to advertised product desire. Positive correlations with materialism at both waves were only found for the overall internet use measure (2.1), which can be seen as an indication that this measure has the highest construct validity. In sum, for internet advertising exposure the broad measure performed the worst and the overall internet use measure performed the best on construct validity.

Discussion

This study was the first to examine survey measures for children’s advertising exposure, differing in their content specificity and to compare their reliability and validity. Taking both factors into account, the results suggest that all measures provided solid estimates for children’s television and internet advertising exposure. Still, as will be explained in the following paragraphs, it may be concluded that television advertising exposure can best be measured by asking children how often they watch certain popular (commercial) network stations, either weighting or not weighting their scores for advertising density. Exposure to internet advertising can best be measured by asking children how often they use the internet or asking them how often they visit certain popular websites, weighting their scores for advertising density. These measures can be used across disciplines (e.g. marketing, communication science or psychology) and in all research methodologies using children’s self-reports, including surveys, experiments and interviews. Thus, these self-report measures can be used in all types of studies taking a consumer behavior perspective toward media advertising, investigating the (unintended) effects of advertising.

In the sections below, practical considerations for applying the measures of children’s advertising exposure, and some concrete advise on the preferred measures are provided for children’s television and internet advertising exposure separately. The text then moves to general suggestions for future research. In the final section, it is explained how the current measures for children’s television and internet advertising exposure can be adapted to measure children’s exposure to newer forms of advertising.

Television advertising exposure

From a practical stance, it may be preferable to measure television advertising exposure as exposure to the medium or exposure to networks rather than programs because broadcasting schedules change. Even within the short time frame of the questionnaire used in this study (i.e. two weeks), two of the television programs that were included in the measure for Wave 1 were no longer being broadcast at Wave 2. Between the first and second waves, a new television season started. Although similar programs are designed to appeal to the same type of audience, their audience reach might differ.

Another reason to prefer the more generic measures is that the more content-specific measures did not really outperform the broad measures in terms of reliability and validity. The criterion validity was optimal across all measures, and enriching the survey data with advertising data did not result in higher test-retest reliability or content validity. Mind though, that even though it is not necessary to use advertising data to enrich the measure of children’s television advertising exposure, it may be the desired strategy if researchers want to examine the effects of exposure to food or alcohol advertising on consumption patterns (Buijzen, Bomhof and Schuurman, 2008; Grenard et al., 2013).

Internet advertising exposure

As was observed in the data set, comparable to the programs children are watching, the websites that children are interested in and the intensity of their use change over time. Therefore, the most stable measure for children’s internet advertising exposure is capturing exposure to the medium (1.1). Caution is needed for using the advertising enriched measure (2.3b), in view of the difficulties experienced in obtaining internet advertising data. It is very important that researchers determine which data they can obtain prior to making the website selection for their scale. Still, taking into account that caveat, the media use to measure and the advertising enriched measures proved to be superior to the website only level measures (2.2 and 2.3a). The media use measure because it scored high on both test-retest reliability and construct validity; the advertising density enriched measure because it scored high on test-retest reliability and criterion validity, and relatively high on construct validity. For this measure, no significant correlation was found with materialism, but this was also true for the website measures.

Although the criterion validity for the media use measure was low, this finding can be explained by the content of the items. This measure refers to children’s overall internet use, whereas the other measures refer to website visits. Future research should explore how to measure exposure to advertising on the internet beyond exposure to certain websites. As Greenberg et al. (2005) indicated, there are many things one can do online as follows: watch movie clips, listen to music, gather (home)work-related information, gather entertainment-related information, play online videos, e-mail, visit chat rooms, visit forums or other activities. Future research should investigate whether just asking children how often they engage in these activities will give a proper estimate for their online advertising exposure.

Notably, with respect to construct validity, three out of four internet advertising exposure measures were not correlated to materialism. Previous research has only examined the relation between advertising exposure and materialism for television advertising exposure, not for internet advertising exposure. The current findings could indicate that children’s internet advertising exposure and materialism may be unrelated. Perhaps, this difference in effects between television and internet advertising can be explained by their difference in format. Television advertising often involves commercials with a narrative that stresses not only the importance of possessions but also the favorable outcomes of consumption, such as happiness and success. Such a persuasive narrative is often missing in website advertising, which often takes the form of a banner. Whether this difference in format causes a difference in effect is an issue that should be explored further in future research. To estimate such differences in effects in the most accurate fashion, advanced structural equation modeling could be used to account for the ordinal nature of the measurement items.

Implications for future research

Importantly, the measures assessing children’s advertising exposure that were presented in this study can easily be adapted to measure advertising exposure through other types of media use than television viewing and internet use. Yet, when doing so, researchers should assess which level of content specificity is deemed appropriate for the new medium. Some of the newer forms of media use bear a strong resemblance with traditional medium use as follows: for instance, on-demand television use and online streaming complement traditional television viewing, whereas app use complement traditional internet use. Future methodological research could investigate whether the appropriate levels of content specificity are similar for similar types of media use.

Investigating the appropriate level of content specificity for children’s advertising exposure through newer forms of media use is important given the rapid changes in the media landscape. For instance, youth are increasingly watching on-demand and online streaming television (Ofcom, 2017). Some of these channels may contain considerable amounts of advertising, whereas others may not. Adapting the current items to refer to the most popular on-demand and online channels (e.g. “How often do you watch […]”) may result in an appropriate measure for children’s advertising exposure through these types of channels. Furthermore, children are also increasingly using mobile devices, such as smartphones and tablets, to access the internet (Ofcom, 2017) and many of the applications on these devices contain advertising (Nairn and Hang, 2012). Adapting the current items to refer to the most popular branded apps (e.g. “How often do you use the […] app”) may result in an appropriate measure for children’s advertising exposure through mobile applications.

Researchers need to conduct pre-studies among present-day child audiences to learn more about their preferred media, media outlets and media content and determine which media, outlets and content need to be included when studying their advertising exposure through newer forms of media. These pre-studies can be conducted in survey form or researchers may opt for studying passive user data made available. Though some researchers have called for combining survey and passive data (Revilla et al., 2017), scholars do need to be aware that it might be difficult to get ethical approval for a study tracking children’s media behavior as this entails the collection of personal and privacy sensitive data. Hence such a study could be costly in money and time. Also, when examining the construct validity of new measures for children’s advertising exposure, it is recommended that research include convergent validation measures suited for the type of advertising at hand. For branded apps, for instance, one could consider asking children about the frequency at which they ask their parents to allow in-app purchases and the frequency at which this leads to parent-child conflict. Finally, when deciding on the preferred level of content specificity, scholars should also contemplate whether they are interested in non-integrated advertising (i.e. commercials in commercial blocks or banner ads), integrated advertising (i.e. product placement or advertorials) or both. With the exception of the measures weighting for advertising density, all other exposure measures in this study carry the advantage that they capture the full scope of children’s advertising exposure.

Hopefully, this study provides researchers with proper tools to start exploring children’s advertising exposure and its effects throughout the full media landscape. Also, by using similar measurements across different types of media (i.e. television, streaming services, web browsers, apps, etc.) scholars can enter a new avenue of research and explore which types of advertising exposure affect children the most. Advertising not only dominates Western children’s life but also is expanding its reach and tightening its grip on children in other regions too (Kasser and Linn, 2016).

Advertising exposure measures

Level of content specificity Television advertising exposure (1.x) Internet advertising exposure (2.x)
Level 1 – media exposure (x.1) 1.1 How often do you watch television (in the morning/afternoon/evening)? 2.1 How often do you spend time on the internet (in the morning/afternoon/evening)?
Level 2 – broad content (x.2) 1.2a How often do you watch the following network channel … (list of both publicly funded and commercial channels)? 2.2 How often do you visit the following website … (list of popular websites)?
1.2b How often do you watch the following television program … (list of programs on both publicly funded and commercial channels)?

Level 3 – specific content (x.3)
1.3a How often do you watch the following network channel … (list of commercial channels only)? 2.3a How often do you visit the following website … (list of popular websites with the most advertisements)?
1.3b How often do you watch the following television program … (list of programs on commercial channels only)?
1.3c How often do you watch the following network channel … (list of both publicly funded and commercial channels) * advertising density 2.3b How often do you visit the following website … (list of popular websites) * advertising density
1.3d How often do you watch the following television program … (list of both programs on non-commercial and commercial channels) * advertising density

Criterion validity

Exposure measure 1.1 1.2a 1.2b 1.3a 1.3b 1.3c 1.3d
Television
Wave 1
1.1 1
1.2a 0.54*** 1
1.2b 0.38*** 0.66*** 1
1.3a 0.58*** 0.90*** 0.62*** 1
1.3b 0.44*** 0.63*** 0.90*** 0.70*** 1
1.3c 0.56*** 0.99*** 0.65*** 0.94*** 0.66*** 1
1.3d 0.38*** 0.60*** 0.88*** 0.57*** 0.88*** 0.61*** 1
Wave 2
1.1 1
1.2a 0.50*** 1
1.2b 0.41*** 0.66*** 1
1.3a 0.51*** 0.90*** 0.60*** 1
1.3b 0.50*** 0.66*** 0.90*** 0.71*** 1
1.3c 0.50*** 0.99*** 0.64*** 0.94*** 0.66*** 1
1.3d 0.43*** 0.64*** 0.94*** 0.61*** 0.91*** 0.64*** 1
Internet
Wave 1
2.1 2.2 2.3a 2.3b
2.1 1
2.2 0.14 1
2.3a 0.12 0.91*** 1
2.3 b 0.11 0.97*** 0.97*** 1
Wave 2
2.1 1
2.2 0.19* 1
2.3a 0.13 0.91*** 1
2.3 b 0.15 0.97*** 0.96*** 1
Notes:

*p < 0.05;

**

p < 0.01;

***

p < 0.001

Construct validity

Exposure measure Gender Age Advertised product desire Materialism
Television
Wave 1
1.1 0.10 0.10 0.49*** 0.37***
1.2a 0.01 0.06 0.48*** 0.24**
1.2b −0.10 0.09 0.43*** 0.13
1.3a 0.08 0.11 0.55*** 0.30***
1.3b −0.11 0.12 0.53*** 0.26***
1.3c 0.05 0.08 0.49*** 0.26***
1.3d −0.13 0.09 0.36*** 0.20*
Wave 2
1.1 0.13 0.14 0.32*** 0.18*
1.2a −0.07 0.03 0.35*** 0.15*
1.2b −0.10 0.03 0.35*** 0.07
1.3a −0.03 0.08 0.39*** 0.19*
1.3b −0.11 0.05 0.40*** 0.14
1.3c −0.04 0.00 0.35*** 0.18*
1.3d −0.11 −0.00 0.29*** 0.10
Internet
Wave 1
2.1 0.11 0.11 0.44*** 0.36***
2.2 0.06 −0.16* 0.32*** 0.10
2.3a 0.06 −0.12 0.30*** 0.10
2.3b 0.07 −0.15 0.33*** 0.08
Wave 2
2.1 0.11 0.11 0.33*** 0.25**
2.2 0.11 −0.14 0.32*** −0.11
2.3a 0.08 −0.11 0.22** −0.13
2.3b 0.11 −0.15 0.26*** −0.15
Notes:

*p < 0.05;

**

p < 0.01;

***

p < 0.001

References

Al Baghal, T. (2014), “Is vague valid? The comparative predictive validity of vague quantifiers and numeric response options”, Survey Research Methods, Vol. 8 No. 3, pp. 169-179, doi: 10.18148/srm/2014.v8i3.5813.

Annenberg Media Exposure Research Group (2008), “Linking measures of media exposure to sexual cognitions and behaviors: a review”, Communication Methods and Measures, Vol. 2 No. 1, pp. 23-42.

Araujo, T., Wonneberger, A., Neijens, P. and de Vreese, C. (2017), “How much time do you spend online? Understanding and improving the accuracy of self-reported measures of internet use”, Communication Methods and Measures, Vol. 11 No. 3, pp. 173-190.

Atkin, C.K. (1975a), Survey of Pre-Adolescent’s Responses to Television Commercials: The Effects of Television Advertising on Children (Report No. 6), MI State University.

Atkin, C.K. (1975b), Survey of Children’s and Mother’s Responses to Television Commercials. The Effects of Television Advertising on Children (Report No. 8), MI State University.

Borgers, N. (2003), “Questioning children’s responses”, Doctoral dissertation, Utrecht University.

Buijzen, M. and Valkenburg, P.M. (2000), “The impact of television advertising on children’s christmas wishes”, Journal of Broadcasting & Electronic Media, Vol. 44 No. 3, pp. 456-470, doi: 10.1207/s15506878jobem4403_7.

Buijzen, M. and Valkenburg, P.M. (2003a), “The effects of television advertising on materialism, parent-child conflict, and unhappiness: a review of research”, Journal of Applied Developmental Psychology, Vol. 24 No. 4, pp. 437-456.

Buijzen, M. and Valkenburg, P.M. (2003b), “The unintended effects of television advertising. A parent-child survey”, Communication Research, Vol. 30 No. 5, pp. 483-503.

Buijzen, M. and Valkenburg, P.M. (2005), “Parental mediation of undesired advertising effects”, Journal of Broadcasting & Electronic Media, Vol. 49 No. 2, pp. 153-164.

Buijzen, M., Bomhof, E. and Schuurman, J. (2008), “Explaining the link between television viewing and childhood obesity: a test of three alternative hypotheses”, Journal of Children and Media, Vol. 2 No. 1, pp. 67-74, doi: 10.1080/17482790701733203.

Buijzen, M., Schuurman, J. and Bomhof, E. (2008), “Associations between children's television advertising exposure and their food consumption patterns: a household diary-survey study”, Appetite, Vol. 50 No. 2-3, pp. 231-239.

Churchill, G.A. and Moschis, G.P. (1979), “Television and interpersonal influences on adolescent consumer learning”, Journal of Consumer Research, Vol. 6 No. 1, pp. 23-35.

de Leeuw, E. (2003), Questioning Children in Surveys (Report to the Statistical Information and Monitoring Programme on Child Labour of the International Labour Office’s Internal Programme on the Elimination of Child Labour), Department of methodology and statistics, Utrecht.

de Leeuw, E., Borgers, N. and Smits, A. (2004), “Pretesting questionnaires for children and adolescents”, in Presser, S., Rothgeb, J.M., Couper, M.P., Lessler, J.T., Martin, E., Martin, J. and Singer, E. (Eds), Methods for Testing and Evaluating Survey Questionnaires, John Wiley & Sons, pp. 409-429.

Diedenhofen, B. (2012), “Cocor: comparing correlations (version 0.01-1) [software]”, available at: http://r.birkdiedenhofen.de/pckg/cocor/

Du Plessis, E. (1994), “Recognition versus recall”, Journal of Advertising Research, Vol. 34, pp. 75-91.

DeVellis, R.F. (2003), Scale Development. Theory and Applications. Applied Social Research Methods Series (Vol. 26), Thousand Oaks.

Greenberg, B.S., Eastin, M.S., Skalski, P., Cooper, L., Levy, M. and Lachlan, K. (2005), “Comparing survey and diary measures of internet and traditional media use”, Communication Reports, Vol. 18 Nos 1/2, pp. 1-8, doi: 10.1080/08934210500084164.

Grenard, J.L., Dent, C.W. and Stacy, A.W. (2013), “Exposure to alcohol advertising and teenage alcohol-related problems”, PEDIATRICS, Vol. 131 No. 2, pp. e369-e379.

Jordan, A., Trentacoste, N., Henderson, V., Manganello, J. and Fishbein, M. (2007), “Measuring the time teens spend with media: challenges and opportunities”, Media Psychology, Vol. 9 No. 1, pp. 19-41, doi: 10.1080/15213260709336801.

Kasser, T. and Linn, S. (2016), “Growing up under corporate capitalism: the problem of marketing to children, with suggestions for policy solutions”, Social Issues and Policy Review, Vol. 10 No. 1, pp. 122-150.

Kunkel, D. (2001), “Children and television advertising”, in Singer, D.G. and Singer, J.L. (Eds), The Handbook of Children and Media, Sage Publications, pp. 375-394.

Moschis, G.P. and Churchill, G.A. (1978), “Consumer socialization: a theoretical and empirical analysis”, Journal of Marketing Research, Vol. 15 No. 4, pp. 599-609.

Nairn, A. and Hang, H. (2012), Advergames: It’s Not Child’s Play. A Review of Research, Family and Parenting Institute.

Noar, S.M. (2003), “The role of structural equation modeling in scale development”, Structural Equation Modeling: A Multidisciplinary Journal, Vol. 10 No. 4, pp. 622-647, doi: 10.1207/S15328007SEM1004_8.

Oates, C., Newman, N. and Tziortzi, A. (2014), “Parents’ beliefs about, and attitudes towards, marketing to children”, in Blades, M., Oates, C., Blumberg, F. and Gunter, B. (Eds), Advertising to Children, Palgrave Macmillan, pp. 115-136.

Ofcom (2017), “Children and parents: media use and attitudes report”, Ofcom report, available at: www.ofcom.org.uk/research-and-data/media-literacy-research/childrens/children-parents-2017

Opree, S.J., Buijzen, M. and van Reijmersdal, E.A. (2016), “The impact of advertising on children’s psychological wellbeing and life satisfaction”, European Journal of Marketing, Vol. 50 No. 11, pp. 1975-1992.

Opree, S.J., Buijzen, M. and Valkenburg, P.M. (2012), “Lower life satisfaction related to materialism in children frequently exposed to advertising”, PEDIATRICS, Vol. 130 No. 3, pp. e486-491.

Opree, S.J., Buijzen, M., van Reijmersdal, E.A. and Valkenburg, P.M. (2011), “Development and validation of the material values scale for children (MVS-c)”, Personality and Individual Differences, Vol. 51 No. 8, pp. 963-968, doi: 10.1016/j.paid.2011.07.029.

Opree, S.J., Buijzen, M., van Reijmersdal, E.A. and Valkenburg, P.M. (2014), “Children’s advertising exposure, advertised product desire, and materialism: a longitudinal study”, Communication Research, Vol. 41 No. 5, pp. 717-735.

Pine, K.J. and Nash, A.S. (2002), “Dear santa: the effects of TV advertising on children”, International Journal of Behavioral Development, Vol. 26 No. 6, pp. 529-539, doi: 10.1080/01650250143000481.

Potter, W.J. (2014), “A critical analysis of cultivation theory”, Journal of Communication, Vol. 64 No. 6, pp. 1015-1036.

Raghunathan, T.E., Rosenthal, R. and Rubin, D.B. (1996), “Comparing correlated but nonoverlapping correlations”, Psychological Methods, Vol. 1 No. 2, pp. 178-183.

Revilla, M., Ochoa, C. and Loewe, G. (2017), “Using passive data from a meter to complement survey data in order to study online behavior”, Social Science Computer Review, Vol. 35 No. 4, pp. 521-536.

Richins, M.L. (2017), “Materialism pathways: the processes that create and perpetuate materialism”, Journal of Consumer Psychology, Vol. 27 No. 4, pp. 480-499.

Rideout, V.J. and Robb, M.B. (2019), The Common Sense Census: Media Use by Tweens and Teens, 2019, Common Sense Media.

Rideout, V.J., Foehr, U.G. and Roberts, D.F. (2010), Generation M2: Media in the Lives of 8- to 18-Year Olds, Henry J. Kaiser Family Foundation.

Romantan, A., Hornik, R., Price, V., Cappella, J. and Viswanath, K. (2008), “A comparative analysis of performance of alternative measures of exposure”, Communication Methods and Measures, Vol. 2 Nos 1/2, pp. 80-99, doi: 10.1080/19312450802062539.

Rozendaal, E., Buijzen, M. and Valkenburg, P.M. (2009), “Do children's cognitive advertising defenses reduce their desire for advertised products?”, Communications, Vol. 34 No. 3, pp. 287-303.

Rozendaal, E., Buijzen, M. and Valkenburg, P.M. (2010), “Comparing children's and adults' cognitive advertising competences in The Netherlands”, Journal of Children and Media, Vol. 4 No. 1, pp. 77-89.

Rozendaal, E., Opree, S.J. and Buijzen, M.A. (2016), “Development and validation of a survey instrument to measure children’s advertising literacy”, Media Psychology, Vol. 19 No. 1, pp. 72-100.

Sargent, J.D., Worth, K.A., Beach, M., Gerrard, M. and Heatherton, T.F. (2008), “Population-based assessment of exposure to risk behaviors in motion pictures”, Communication Methods and Measures, Vol. 2 Nos 1/2, pp. 134-151.

Sikkema, P. (Ed.) (2009), Jongeren 2009., We Laten on, s niet gek maken. [Youth 2009., We won’t let ourselves be driven mad] Qrius.

Sikkema, P. (Ed.) (2012), Kinderen en Jongeren., Positieve Kracht., [Kids and Youth, Positive Forces], Qrius.

Slater, M.D. (2004), “Operationalizing and analyzing exposure: the foundation of media effects research”, Journalism & Mass Communication Quarterly, Vol. 81 No. 1, pp. 168-183.

Van der Voort, T.H.A. and Vooijs, M.W. (1990), “Validity of children’s direct estimates of time spent television viewing”, Journal of Broadcasting & Electronic Media, Vol. 34 No. 1, pp. 93-99, doi: doi:10.1080/08838159009386729.

Vooijs, M., Van der Voort, T. and Beentjes, H. (1987), “De geschiktheid van verschillende typen vragen om de kijktijd en leestijd van kinderen te meten: een validatie-onderzoek”, [the Applicability of Different Measures for Children’s Screen and Reading Time: A Validation Study], Massacommunicatie, Vol. 15, pp. 65-80.

Vandewater, E.A. and Lee, S.-J. (2009), “Measuring children’s media use in the digital age. Issues and challenges”, American Behavioral Scientist, Vol. 52 No. 8, pp. 1152-1176.

Ward, S. and Wackman, D. (1971), “Family and media influences on adolescent consumer learning”, American Behavioral Scientist, Vol. 14 No. 3, pp. 415-427.

Wonneberger, A. (2011), “Exposure to public-affairs TV in a changing viewing environment”, Doctoral dissertation, Amsterdam School of Communication Research.

Corresponding author

Suzanna Opree can be contacted at: opree@eshcc.eur.nl

About the authors

Suzanna Opree is based at the School of Communication and Culture ERMeCC, Erasmus Research Centre for Media, and Erasmus School of History Culture and Communication, Erasmus Universiteit Rotterdam, Rotterdam, The Netherlands.

Moniek Buijzen is based at the Erasmus School of Social and Behavioral Sciences, Erasmus University Rotterdam, Rotterdam, The Netherlands.

Eva van Reijmersdal is based at the Amsterdam School of Communication Research, University of Amsterdam, Amsterdam, The Netherlands.

Related articles