Lies, Damn Lies and Statistics
A former British Prime Minister is credited with the above quote … and I often sympathize with him.
I used to be a Maths teacher … and so figures don’t frighten me as much as they do many other people … but how people use figures does worry me. There is something about figures that gives them authority … or rather, gives an authority to the person quoting them: “You can’t argue with the figures.”
On the contrary, you can – and you SHOULD!
There is the joke – I presume it IS a joke – about the museum guide showing tourists around and when she came to the skeleton of a dinosaur, she proudly announced that the museum staff had given the creature the nickname “Lucy”, and that Lucy was “two million and eight years old”. Once, when she was showing a school party around, she made her usual speech – and the teacher accompanying the group was astounded by this news, and asked how she knew the age of the skeleton. She replied: “Well, when I first came to work here they told me that the skeleton was 2,000,000 years old. That was eight years ago … so, now she must be 2,000,008.”
It was the Archimedean School Sometimes the reverence in which the authorities are quoted here in the Kyrgyz Republic reminds me of the Archimedean School of ancient Greece which used to consider numbers as ‘divine’. In this modern age – we should know better!
As someone who has had to teach Statistics, I know that they can be very useful as a management tool, as an indicator, but I also know that Statistics raise more questions than they answer. For example: Where did these figures come from? , What do they actually mean?, What are they being used to say?, Do they really say that?, Do they really make sense? … and so on.
And that is just looking at the figures … not who is quoting them, their motivation – what they are trying to prove, … and so on.
There are many different examples of “strange” statistics. To give just two examples:
I remember, when I was young, being told that the average British family had 2.4 children … so, apart from being left with the question “what does .4 of a child look like?”, I was left with the feeling the I didn’t come from an average family – we only had two children, (my brother and I). We may not have been average in all sorts of ways – but I can’t help feeling that in others, we were exactly that. As a teacher, whenever I had to teach averages, I would ask the class how many children came from an “average” family and in response to this question a number of children would raise their hands. When I asked, which part of the body – an arm, a leg, the head, represented the 0.4 of the child in their family there were smiles all round – and I hope they learned the lesson to treat statistics with a certain amount of caution – suspicion, even.
Recently it was reported that according to the Official statistics, Kyrgyzstan bought goods and services worth some USD84 million from the People’s Republic of China – but if you look at the Chinese figures – they sold goods and services worth USD145million. How can we reconcile these figures? Actually, I don’t think it is that surprising – if a large part of the Kyrgyz economy really is in the “shadow” as many people say it is.
The other day I was invited to an interview with a group of foreign experts who were in the Kyrgyz Republic to investigate the Tourist industry. I told them, “Don’t believe the official statistics” – and the Minister for tourism was sitting next to me at the time … actually, I think he would probably agree with me.
We have to ask ourselves many different questions about the statistics. For example: where did they come from?, what is the source of the figures. Let’s consider an example: how many tourists visited the Kyrgyz Republic? … According to the official statistics, in 2002 there were 139,600 tourists; in 2003 – 211,300; and in 2004 – 1,527,500. This seems to show pretty impressive growth … BUT, it should be noted that the sources used for the 2004 figure was different. For the first two years, the figures are taken from the State Statistical Committee and are based on reports made to them by tour companies, hotels end so forth, of the number of clients. The 2004 figures were obtained from the Border Authorities – and reflect the number of foreigners crossing the country’s borders – tourists, businessmen, diplomats … and so on. If you take the figures of the National Statistics Committee for 2004 – then (for the first nine months of the year) there were only 703700 tourists – still an impressive figure … but well short of, (indeed, less than half of …), the 1,411,190 reported by the border authorities for the same period. How do we reconcile THESE differences? (Needless to say – these statistics have recently been the subject of considerable controversy.)
The State Statistical Office receive figures from companies, hotels etc. – but, how many tourists from neighbouring Kazakhstan, Uzbekistan and Russia come independently, without using a tour company, or stay with family and friends … – and how many “Western” tourists as “independent travelers”? The border authorities report the number of people crossing the various borders – but how do we know who is coming as a tourist and who is coming on business. (In Mongolia, for example, officially there are a large number of Chinese tourists every year … but in fact many of them are market traders – who use a tourist visa because it is cheaper and less bureaucratic … such ‘Shopping Tourism’ is quite big business here as well. For British readers – it may be similar to the shopping trips over the Channel – to take advantage of cheaper prices for wine etc. in Calais and Boulogne).
Each month we have to give a report from our hotels to the State Statistical Committee about how many have stayed with us; from which country; how long they stay with us; how much they spent for accommodation; and the purpose of their visit. The first four are quite easy for us to collect … we have to keep a register and accounts, and it is just a bit of extra clerical work … but how do we get the reason for their visit. We actually ask them, there is question on the registration form – but more often than not this question is ignored. In the end, in most cases, we go by their visa – assuming they need one – and rely on the consular official having done his job properly.
Of course, the government are concerned that much of the tourism sector operates in the “shadow economy” – not reporting and not paying tax. There may be some truth in that – but is the problem as big as is claimed. The Prime Minister says he calculates that the tourist business in Issyk Kul is worth about USD100 million a year – but only something like USD32 million is reported. Actually, he says that the budget receives only USD32 million – but the budget (i.e. the money raised through taxes) receives far less. The tax revenue should be about a third of all the turnover – IF the regular tax regime is used throughout. Unfortunately, Kyrgyzstan has a number of different tax regimes which complicates the issue even further – but that’s another story – and maybe another “postcard”. Back to Statistics.
Another problem with statistics is the analysis – the way in which the original, raw, data is processed. I remember hours of study into different types of test which could be applied … depending upon things like the type of data (nominal, discrete or continuous), the population from which the sample was drawn, the size of sample, the type of operation (e.g. comparison of populations or samples, if they were “tied” or not … and so on.) The choice of the wrong test can lead to interesting, and conflicting, results.
Then there is the question of presentation – the way statistics are used. For those who know the difference between a bar chart (where the length of the bar is important) and a histogram (where the area is vital) and a three-dimensional pictogram (where it is the volume of the image presented) … will appreciate that it is easy to create a misleading impression. The same can be said for tables of figures – or even a single figure taken out of context.
Some statistics are based on assumptions … and bear little relationship to what is actually happening in the real world.
For example: as a schoolteacher, I was involved in a pay dispute back in the 1980’s. At a Parent-Teacher meeting which had been called to discuss the issue, one parent asked the question “what is the average teacher’s salary?” For some reason, most of my colleagues didn’t want to answer. Remember, I am not afraid of figures, so I stood up and I said: “It depends which figures you believe … the figure quote in the press is, (Sorry – I forget, now, but it was quite a high sum and there were sharp intakes of breath around the room) … but can we believe this?” I went on to point out that this figure was simply taken because it was the top of “Scale 2” – which meant that a third of all school teachers who were on Scale 1 could not ever hope to achieve this even after years in the profession, for another third it was the maximum that they could hope to achieve. For the other third … some, senior staff, would be earning more than this – but not all – as the scales overlapped. There may well have been reasons why this figure was chosen as an “average statistic” – but it didn’t really say much about the money being earned by the ‘average’, individual, classroom teacher.
(Please note I have been out of teaching for ten years – as I understand it, the system has changed … and this might not reflect the current situation – I quote it to underline that behind every statistic there is a story … and we should try to understand the full story as best we can).
To go back to the Prime Minister and his calculations concerning the tourist industry … I heard him in one meeting saying … “and how much does a typical foreign tourist pay per day – say, USD100? …” I wish they did – they don’t pay us that much when we organize a tour for them. In 2004 the ‘average’ sum paid to us by tourists was only USD91 a day, (about USD22 of that was tax!). OK – that’s pretty close to USD100 – but some recent market research asked the tourists themselves how much they spent and almost 30% of “Western” tourists said that they spent up to USD100 PER WEEK – not per day … about 23% said between USD100 and USD200, another 21% said between USD200 and USD350, just over 11% said between USD350 and USD500 … with ONLY 15% saying over USD500. Of course, if we assume that they are spending “say, USD100” a day … that makes USD700 per week – and most of them should have fallen into this final category.
Once again … how do we reconcile these figures?
After all this, you might be justified in thinking that I don’t take much notice of statistics. That’s not true. In fact, I spend a lot of my time collecting and analyzing statistics. I think they are a valuable management tool. They help keep me informed about what is going on, trends which I can try and extrapolate to the future and on which I can base decisions. But they are just that – tools … they are not gods, they are not infallible … and in the end it is I that has to make a decisions, and carry the responsibility. Figures need to be treated with respect and understanding … and a little distrust and a healthy disrespect. As I said earlier: “statistics raise more questions than they answer”.
Bishkek