35 replies to this topic

[alanh]

Goformoe,
Have you filled out a survey form? The one for Geelong will not include any digital stations anyway.
This is all supposition no provable facts.

[Malich]

alanh, on 06 May 2013 - 02:20 AM, said:

This is all supposition no provable facts.

Yup.

On the other hand, GoForMoe's discussion is quite thoughtful, thought-provoking, and interesting.

[GlennP]

For Alanh (& anyone else interested but don't know how it works).
This is how the ratings diaries are setup.

From Nielsen.

Quote

A diary is used by each individual in the survey. Each diary contains adequate space for the individual to record all his or her radio listening, both at home and away from home, by placing crosses in appropriate squares for every quarter-hour of listening (eight minutes or more) 24 hours a day. Stations, quarter-hours, day and place of listening for a period of 9 days commencing on a Friday record the data. The actual days processed are Sunday to Saturday.

Metropolitan Markets
Each diary has the quarter-hours listed along the side for each day. A set of stickers listing station names are included with each diary. Respondents are to peel of the sticker for each station listed to and place it in columns in the diary. Up to seven columns are provided for stickers in each diary. Blank stickers are available for respondents to write in any station listed to that is not on the pre-printed list. Additional pages are provided in the diary to record listening if more than 8 stations are listened to. Respondents also record where they listened – at home, in the car, at work or elsewhere.
There is also provision in the diaries to indicate the device used to listen (AM/FM radio, Internet and Digital radio, as well as if via a Mobile Phone or Tablet).

Regional Markets
Stations are pre-listed in the diary across the top and bottom of each page and quarter- hours are listed down the side. Stations are listed in descending order of frequency in half of the diaries and in ascending order of frequency on the other half. This minimises any "order of listing" bias. Two additional columns are included in each diary for recording listening to any other stations (FM & AM) not listed in the diary. Respondents are asked to write in the call signs of such stations. Respondents also record where they listened – at home, in the car, at work or elsewhere.

So in conclusion, all stations listened to are recorded, (even out of market/licence area stations), data is tabulated, & a whole pile of demographic data is produced along with listening data (ratings).

Stations not paying for the data aren't listed in the public released "ratings figures". The figures for all stations, is however available to those who pay, (if they want them).
Digital radio listening figures are collected separately, along with, on what device you listen to, this is all available privately too.

Digital figures aren't available publicly, because as CRA have stated "there's not enough data".
Why is there not enough data?
Because there's not enough/many listeners, & the figures wouldn't be statistically large enough to be anywhere near correct, & would likely be embarrassing in the least.

Like GoForMoe said, if the figures for DAB+ were good/very good, the stations would be shouting them from the rooftops & letting everyone know.

Just for Alanh's information, (& to prove he doesn't know what he's talking about & his wild claims are incorrect).

The Sydney Metro ratings survey area, includes the entire adjacent Gosford radio licence area as well, & the Melbourne Metro ratings survey area, also includes the Geelong radio licence area.
So if GoForMoe was filling out a survey form, his data would actually be for/included in the Melbourne radio ratings, & he would have DAB+ digital radio stations included in his diary.
If you still don't believe me, I can post a copy of the actual survey area map, from Nielsen.

Alanh, you're the only one making supposition with no provable facts.

Edited by GlennP, 06 May 2013 - 10:43 AM.

[lizclinton]

I think a telling sign is the almost total lack of advertising on the commercial digital only stations. If thousands were actually flocking to these new stations then surely after all these years advertisers would have caught on and would be busily pushing their products and services to all those ears.

[Malich]

GlennP, on 06 May 2013 - 09:02 AM, said:

Just for Alanh's information, (& to prove he doesn't know what he's talking about & his wild claims are incorrect).

The Sydney Metro ratings survey area, includes the entire adjacent Gosford radio licence area as well, & the Melbourne Metro ratings survey area, also includes the Geelong radio licence area.
So if GoForMoe was filling out a survey form, his data would actually be for/included in the Melbourne radio ratings, & he would have DAB+ digital radio stations included in his diary.
If you still don't believe me, I can post a copy of the actual survey area map, from Neilsen.

Indeed. I have some maps myself, but wasn't sure of their currency or the legality of posting them (they were legally obtained a while ago, but for a specific purpose), so I didn't post them earlier. I can confirm everything that GlennP says above is true (as it should be - I gather he's in a better position to know than I am).

I also see now that (crappy, but readable) survey maps for all capital cities are freely available as part of a supplementary download from the Nielsen website. These too show GlennP's statements above to be true.

Alan, you don't have to take GlennP's or my word for any of these facts - but remember: "I don't have to prove it to you. You can find it for yourself."

And, as I've said before: the CRA's claims, statements, and figures in their published "industry reports" don't stand up to even rudimentary examination. Anybody who takes them at face value is being badly misled.

[GoForMoe]

alanh, on 06 May 2013 - 02:20 AM, said:

Goformoe,
Have you filled out a survey form? The one for Geelong will not include any digital stations anyway.

If I moved to within the bounds of the Melbourne licence area and filled out a survey would you then respond to the points being made?

[DX Fan]

I disagree with the conclusions stated that "Listening to a station on DAB+ doesn't count to the AM/FM ratings". My reading of the Nielsen Radio Survival Kit suggests to me that all ways of listening to a participating radio station are recorded and reported in the published ratings. There is "provision in the diaries to indicate the device used to listen" but it doesn't say that these figures are not added in. It explains how each diary has "adequate space for the individual to record all his or her radio listening." (my emphasis) in the grid system descibed above. The only data that is not published in the ratings is "OTHER AM, OTHER FM and OTHER DIGITAL" (again my emphasis).

This strongly suggests to me that the published rating for a station included those listening by whatever means - AM, FM repeater, DAB+ or internet, phone app etc.

[GlennP]

DX Fan, on 06 May 2013 - 08:03 PM, said:

I disagree with the conclusions stated that "Listening to a station on DAB+ doesn't count to the AM/FM ratings". My reading of the Nielsen Radio Survival Kit suggests to me that all ways of listening to a participating radio station are recorded and reported in the published ratings. There is "provision in the diaries to indicate the device used to listen" but it doesn't say that these figures are not added in. It explains how each diary has "adequate space for the individual to record all his or her radio listening." (my emphasis) in the grid system descibed above. The only data that is not published in the ratings is "OTHER AM, OTHER FM and OTHER DIGITAL" (again my emphasis).

This strongly suggests to me that the published rating for a station included those listening by whatever means - AM, FM repeater, DAB+ or internet, phone app etc.

DAB+ digital radio figures aren't allowed to be publicly published or discussed, with anyone outside who ever has them, CRA embargo, not Nielsen.
Other digital, means, things like internet listening to a station, out of market, or international, or an internet only station.

Did you read my quote from CRA in post #18? here it is again.

Quote

The radio industry will look at releasing individual digital radio station audience figures when there is enough data to establish trends.

Ratings figures are found statistically using small survey numbers, (you would've seen them in the survival kit). 2400 surveyed in Sydney for just over 4 million people, & 2400 surveyed in Melbourne for just over 4 million people, (smaller survey's in smaller markets).

With the small amount of people surveyed who would indicate they listen to DAB+, you can't give any sort of statistical accuracy to the results, especially when not all "potential" listeners even own a DAB+ radio.

Let's say for example, 10% of those surveyed, indicate they listen to 2DAY on DAB+. You can't equate/add that to the figures, because 10% of Sydney listeners, mightn't/don't own or have access to a DAB+ radio.
You can't just say, well, those people who would/should, be listening to 2DAY on DAB+ (but can't), would/should be listening to 2DAY on FM anyway. Statistically that would be incorrect, you can't prove that, because that's not what the survey results said, it's only an assumption.

At one stage early on in the game, CRA did publish overall nationwide listener figures, that were something like over 1 million listeners, when only 400,000 DAB+ radio receivers had been sold.
So that meant, there was more than 2 people listening to every single DAB+ radio in Australia, (car radios & HiFi radios weren't included).
Those figures were listeners for the small portable/kitchen style & personal walkman style DAB+ digital radio receivers, which would be highly unlikely & near impossible.
They no longer publish figures like that, just percentage figures. But a percentage of what, no one but them knows?

All figures are accounted for individually, & the stations know "privately" all the individual breakdown figures of how many listen & on what platform they listen.
Public ratings figures are AM/FM listeners only, which is still over 90% of the total overall listening figures. Or to put it another way, roughly 90%+ of all people listening to radio, still listen (mainly/only) on AM or FM.

Edited by GlennP, 07 May 2013 - 08:45 AM.

[Malich]

GlennP, on 07 May 2013 - 08:39 AM, said:

Ratings figures are found statistically using small survey numbers, (you would've seen them in the survival kit). 2400 surveyed in Sydney for just over 4 million people, & 2400 surveyed in Melbourne for just over 4 million people, (smaller survey's in smaller markets).

To put on my statistician's hat & disagree a bit: those survey sizes aren't small at all - from a population statistics PoV, a sample size of 2400 out of a population of 4,000,000 represents quite an intensive sampling rate. Without actually running the numbers (and I might later; it's probably taking me longer to type this than to actually do it), and assuming the worst-case, I'd expect results to be accurate to within 2% @ 95% confidence.

That is: for example, if in reality 25% of radio listeners listen to (e.g) 2DAY-FM, the survey has a 95% chance of reporting between 23% and 27%.

Counterintuitively, the more stations there are and the more fragmented the market (i.e. less listening to each station), the audience figures may be more accurate than that. (Up to a point - you run into the problems when your effective sample size becomes too small to accurately capture less common events e.g. listening to smaller stations). Given that there are many stations in a market, and throw in demographics, and I'd bet that accuracy is more like ±1% for all bar the smallest stations.

GlennP, on 07 May 2013 - 08:39 AM, said:

With the small amount of people surveyed who would indicate they listen to DAB+, you can't give any sort of statistical accuracy to the results, especially when not all "potential" listeners even own a DAB+ radio.

And that's the crunch - given the BS figures in the CRA reports & their reluctance to release any real numbers, I suspect that DAB+ stations fall into the "all bar the smallest stations" category I mentioned above. Based on my estimates above, my gut feeling (again, not calculated, but I might check the CRA reports to see if enough info can be gained to do some analysis) is that DAB+ listeners represent something less than 5%, possibly 10%, of the surveyed population.

GlennP, on 07 May 2013 - 08:39 AM, said:

Let's say for example, 10% of those surveyed, indicate they listen to 2DAY on DAB+. You can't equate/add that to the figures, because 10% of Sydney listeners, mightn't/don't own or have access to a DAB+ radio.

You can, however, factor it in if the survey captures other info - whether participants own a DAB+ radio, where it is (e.g. home/office/car), records where / which mode is used for listening for each entry, etc.

GlennP, on 07 May 2013 - 08:39 AM, said:

You can't just say, well, those people who would/should, be listening to 2DAY on DAB+ (but can't), would/should be listening to 2DAY on FM anyway. Statistically that would be incorrect, you can't prove that, because that's not what the survey results said, it's only an assumption.

True - however, interesting / important information could be gleaned by comparing listening habits across devices (e.g. % of DAB owners who listen to 2DAY on DAB+ in the home vs listen to 2DAY on FM in the car, etc). To get that, as I mentioned above you'd have to report different modes of listening (FM / DAB+ / online) separately. Does the survey record that info? I don't know...


(Edit: And, to follow up with some calculated estimates of accuracy...

Assuming a sample size of 2400 from an estimated population of 4 million:

• Worst-case scenario : exactly ±2% @ 95% confidence (good guess, huh!)
  
• Estimated accuracy for Sydney market overall, factoring in the 14 stations in the Nielsen ratings list : ~±1% @ 95% confidence (another good guess!).
  
• Estimated accuracy for 2DAY-FM (9.3% of the total P10+ share in 2013/1): ~ ±0.4% @ 95% confidence (Note: I originally miscalculated this as "~1.1% @ 95% confidence" - I treated the 2DAY's figure as being 9.3% of the 4 million potential audience, when in fact it's of the ~550,000 actual audience. As the survey results only report the top 14 stations, which only add up to (IIRC) ~83% of total listeners, the true accuracy will lie between ±0.4% and ±1.1% - probably closer to the former. I could estimate something closer, but it would be an estimate on top of an estimate...)

Note that none of that accounts for demographics, listening location, etc - which, largely, will tend to improve accuracy...)

Edited by Malich, 07 May 2013 - 12:06 PM.

[GlennP]

Malich, on 07 May 2013 - 10:51 AM, said:

True - however, interesting / important information could be gleaned by comparing listening habits across devices (e.g. % of DAB owners who listen to 2DAY on DAB+ in the home vs listen to 2DAY on FM in the car, etc). To get that, as I mentioned above you'd have to report different modes of listening (FM / DAB+ / online) separately. Does the survey record that info? I don't know...

I agree with your figures & statements, but the sample size is still small, & the DAB+ sample size as a percentage of that is even smaller. CRA have admitted, there's currently not enough data. If there's not enough data to make accurate results, separately, you can't add them to the AM/FM figures.
If my calculations are correct (& they're probably not), the percentage surveyed in Sydney is 0.057% total pop, compared with Canberra who survey 0.299% of total pop, & Newcastle survey 0.307% of total pop. What I'm saying is if the percentage surveyed was larger, you'd have more accurate results for right down to the smaller stations & even small listener figures for DAB+.

We don't even know where the DAB+ radio sales come from? Thousands of these would be sold in regional areas that can't & may never receive DAB+ radio. If they're included, actual metro units will be less, so that makes less units for potential listeners. how do you factor in potential DAB+ listeners? DAB+ units could also be bought regionally & used in metro areas or vice versa. Yes you can glean actual DAB+ listeners roughly from the surveys, but you can't get percentages as you don't know what the potential actually is. Thus some of the wild claims that have been made, such as what I posted before, over 1 million listeners, but only 400,000 DAB+ radios sold/used. It's a rough guesstamate, not statistically accurate measurements.

As for your quoted question, to the best of my knowledge, all these are included & collected separately in the surveys & stations know these figures. Each station knows how many people of what demographic listen to their station on what platform, where they listen & for how long. Those figures, along with the DAB+ figures are not publicly released.
That's the whole point of this argument. CRA make wild claims of huge DAB+ listeners, but no one else knows exactly how many it actually is, or can undeniably publicly investigate, calculate figures (publicly audit), & agree with them & say yes, DAB+ is doing well, (or not).

I haven't seen actual DAB+ listener figures (& couldn't say what they are if I had), but know people who have, & can honestly say from what they've told me.
That most stations don't care about DAB+ digital radio ATM because no one listens.
From a programming & technical point of view, DAB+ transmissions are not a high priority, (short/medium term). What goes to air is mainly testing of what they can do, (how many stations they can fit in & how low bit-rates they can put out what people will accept, etc), & because they can & have the free spectrum there to use. If a DAB+ station goes off air, there's no rush/priority to get it back on. If an AM/FM station goes off air, it's priority No.1 & you'll have techs working on restoration within minutes.

Edited by GlennP, 07 May 2013 - 01:16 PM.

[Malich]

Hi GlennP,

First up, let me say I'm 100% in agreement with you about DAB+ listening figures - they're likely so low that they don't even appear above the sampling noise or, if they do, they're just embarassing.

That said, statistics don't quite work like that, and it's a bit of a hobbyhorse of mine (and sometimes I even get paid to ride it!), so...

GlennP, on 07 May 2013 - 12:51 PM, said:

I agree with your figures & statements, but the sample size is still small ... If my calculations are correct (& they're probably not), the percentage surveyed in Sydney is 0.057% total pop, compared with Canberra who survey 0.299% of total pop, & Newcastle survey 0.307% of total pop.

First up, I agree more or less with your numbers for the % surveyed. That said, % of population surveyed is near irrelevant to the statistical accuracy. The population size and sample size individually are much more important, and the relationship between each is not direct (or even necessarily linear...). Everybody naturally assumes that the higher the % sampled, the better - but that's not the case.

(I've done my calculations in R, using a typical sample size algorithm. For anyone wanting to check my figures, Google "sample size calculator", or use the ones here. I assume those calculators will give the same results; to be honest, I've only checked the Sydney figure on it...)

To illustrate, using your figures:

• Sydney - survey size 2400 / pop 4182000 = 0.057%, but estimated overall accuracy is ±2% @ 95% confidence.
  
• Newcastle - survey size 1500, pop 489000 = 0.307%, but estimated overall accuracy is ±2.5% @ 95% confidence.
  
• Canberra - survey size 1100, pop 367200 = 0.299%, but estimated overall accuracy is ±3% @ 95% confidence.

As you can see, Sydney has the highest overall accuracy despite having the lowest sampling intensity by far...

GlennP, on 07 May 2013 - 12:51 PM, said:

What I'm saying is if the percentage surveyed was larger, you'd have more accurate results for right down to the smaller stations & even small listener figures for DAB+.

Agreed - but it's a law of diminishing returns and, as I said above, the relationship between sampling intensity (% of total sampled) & accuracy is not particularly obvious. The actual sampling is the most expensive part of a survey by far, in terms of time, effort, and money. Crunching the numbers after they've been colected is cheap and easy

GlennP, on 07 May 2013 - 12:51 PM, said:

Yes you can glean actual DAB+ listeners roughly from the surveys, but you can't get percentages as you don't know what the potential actually is. Thus some of the wild claims that have been made, such as what I posted before, over 1 million listeners, but only 400,000 DAB+ radios sold/used. It's a rough guesstamate, not statistically accurate measurements.

Speaking of which, I've made some rough guesstimates of my own, based on CRA's own reported data and making a few best-case assumptions (e.g. all DAB+ radio sales are within metro survey areas, sales in each city are proportionate to population, all are used for DAB+ listening only, etc):

(Warning: any statisticians passing by will rightly criticise this, as it's really based on guesswork and supposition. But to them I'll say "that's all we've got to work with, I'm not claiming it's right, and it's only for ballpark estimates anyway...")

CRA 2013 Digital Radio Report:

• "Almost 1.5 million people ... listen on a DAB+ digital radio each week"
  
• "Nearly 1.2 million digital devices have been sold since August 2009"

(Note they they didn't say "DAB+ radios", or even "DAB+ capable devices". But for the sake of argument I'll assume they meant DAB+ because, if anything, it'll bias the results more favourably towards DAB+ listening.)

So, let's assume assume the best - they were all sold within metro survey areas with DAB+ coverage, sales in each are proportionate to Nielsen radio audience size, all are used for DAB+ listening only, etc. I'll also assume that everybody in the Sydney metro radio market owns 1 radio, either AM/FM or DAB+.

• Total listening population across the 5 majors + Canberra & Darwin (Nielsen 10+ audience figures + ABS total pop data for Darwin) = 13519300
  
• Sydney's proportion of DAB+ radios = (4182000/13519300) = 0.334
  
• Number of DAB+ radios in Sydney = 1200000 x 0.334 = 400800 radios
  
• At absolute best, this represents about 1 in 10 radios in Sydney

Now, how hard do we need to sample to estimate DAB+ listeners?

• The highest-rating station in Sydney is 2GB; overall market share is 12%. Let's assume that they're representative of the market overall, and so share the same % of DAB+ radios.
  
• DAB+ share of 2GB's audience = 1.2% (Yes, I know this assumes DAB+ listener figures are rolled into the published overall audience figures - Nielsen & CRA don't say whether it is or not; I'm assuming it is. It makes vanishingly little difference either way...)
  
• Very rough rule-of-thumb, since we don't have any other info: to capture that, we need a confidence interval equal to or less than 1/4 of the expected value. That is, 1.2% / 4 = 0.3%

Therefore, to accurately determine the DAB+ ratings of the overall most popular station in Sydney, you'd need a survey size of 104056.

Or, in other terms, about 40x larger than the current survey size...

Now, that said: while current survey size of 2400 is nothing like large enough for CRA to accurately estimate DAB+ station ratings, it maybe is enough to get an estimate of DAB+ market penetration...

Edited by Malich, 08 May 2013 - 03:42 PM.