Recently, a report came out (“Waar een klein land groot in kan zijn”) by Pieter Tops, Judith van Valkenhoef, Edward van der Torre and Luuk van Spijk (all employed by the Dutch Police Academy) drawing two sensational conclusions. First, that every year, the Netherlands produces € 18 916 882 439 (18,9 billion) worth of synthetic drugs. Second, that in the Netherlands, every year 194 313 376 ecstasy pills are used. This latter conclusions is clearly wrong, which sheds doubt on the veracity of the former. Both the original report and the abbreviated version are written in a sensationalist tone; might the authors have neglected to verify their impressive conclusions with sufficient rigor?
In the Netherland, the report has already been critized extensively. Dutch size Legalize posted an analysis (in Dutch), and Dutch newspaper de Volkskrant published a discussion of the report that concluded that the report is a strong argument for legalisation and regulation. These critisicms make sense: the report urges criminalisation, intensified police activity, and sanctioning as if these are goals rather than means. This is perhaps not surprising given the origin of the report in the police academy; nonetheless one hopes that the police force primarily aims to make itself redundant, instead of keeping itself occupied for the sake of being occupied.
The erroneous estimates
In table 3.4 at page 41 of the report, the authors estimate that in 2017, 971 566 879 ecstasy pills were produced in the Netherlands. They assume 157 milligrams of MDMA per pill (and a street price of € 3,50 per pill in the Netherlands; see supplement 1 on page 239 for more information). In table 3.6 on page 43 of the report they state that 20% of the ecstasy pills that are produced in the Netherlands are also being used in the Netherlands (and therefore, that 80% is being exported) and that therefore, annually, 194 313 376 ecstasy pills are being used in the Netherlands.
If every Dutch resident would use ecstasy, then everybody should use 194 313 376 / 17 000 000 = 11 pills per year. This seems unrealistic in itself, but perhaps there are a few users who use so much that it’s still possible that a smaller group of users uses almost 200 million pills per year. Fortunately, data on use frequencies are available.
How often do ecstasy users use ecstasy?
There are a number of sources on use frequencies of ecstasy users. In this post, I’ll use the ‘Large Nightlife Study’ (het Grote Uitgaansonderzoek, by a Dutch knowledge center, the Trimbos Institute), because this is discussed in chapter 12 of the report, and the data from the samples of the Party Panel rounds of 2015, 2016, and 2018, because there we collected very detailed frequency information.
As mentioned on page 196 of the report, the best available estimate of the number of ecstasy users in the 16-65 year-old population is 2.9%, which yields roughly 390 000 people in the Netherlands.
It is important to emphasize that the Large Nightlife Study and Party Panel were not designed to sample representative samples. Recruitment occurred mostly online, and participants will often have had expectations regarding the questionnaire content in advance. Therefor, over of under estimates of substance use prevalence are likely; and this also goes for estimates concerning frequency and intensity of use. In the Large Nightlife Study and Party Panel rounds from 2015 and 2016, overestimates seem likely (in these studies, participants likely knew that the survey concerned substance use, so it makes sense that the questionnaire was mostly interesting for participants with an interest in substance use), and in the Party Panel round in 2018, underestimates seem plausible (this round concerns flirting and the crossing of sexual boundaries, and a larger proportion of the participants was female; women use less, and because this topic’s link with substance use is weaker, there was less selection for substance-using participants).
In the Large Nightlife Study (page 197 of the report), of those who used ecstasy in the past year, it is estimated that 18% used once, 53% a few times, 19% once a month, 9% a few times a month, 1.1% once a week, and 0.3% a few times a week (nobody used daily or almost daily). In the 2015 Party Panel round (about using highly dosed ecstasy and getting ecstasy tested) these percentages were, respectively, 19.9%, 49.4%, 16.1%, 13.4%, 0.7%, and 0.5%. In the 2016 Party Panel round (about visiting the first aid in a nightlife setting), these percentages were 23.5%, 46.9%, 13.8%, 13.8%, 1.8%, and 0.2%. Finally, for the 2018 Party panel round (about flirting and sexual boundaries), these percentages were 35.6% for using once a year, 43.8% for using a few times a year, 11.0% for using once a month, 6.8% for using a few times a month, and 2.7% for using a few times a week.
These estimates seem relatively homogeneous. This means that either roughly the same selection occurred in each sample, or little selection occurred with respect to use patterns, which would mean that the samples can be considered relatively representative with regards to that variable. In any case these samples seem comparable, and so we can use any of the samples to make further estimates (keeping in mind that we may end up with estimates that are too high).
We will further assume that all Dutch ecstasy users have the same use patterns as the nightlife patrons. Of course, there are many caveats here; but the likelihood that these samples happen to use exceptionally little ecstasy is low.
In Party Panel, the use frequencies weren’t measured in categories, but exactly: participants indicated how often in the past half year they used ecstasy (per week, per month, or in total). Because we have these exact numbers, we can now calculate how many pills are being used in the Netherlands in a year if we make one last assumption: the number of pills per occasion. I will use the Party Panel 2015 data here, which are freely available at https://partypanel.nl/resultResources/15.1/report.html (the table shown below is in the tab “Wave Specific Analyses”, below heading “XTC consumption in pills per year per person and in total”).
Instead of making an assumption, I have calculated the total number of pills per year assuming that 1 to 10 pills were used per occasion. In addition, I calculated how many milligrams of MDMA somebody would use per occasion, again assuming 157 milligrams of MDMA per pill. I then computed the average use frequency per year and multiplied this with the number of pills per occasion to obtain the number of pills per person per year. Then, I multiplied this with the number of users in the Netherlands (390 000) to compute the total number of consumed pills per year. Finally, the final columnshows the total number of pills produced in the Netherlands assuming that 80% is being exported (i.e. five times the total consumption in the Netherlands). Pills per occasion Milligrams MDMA per occasion Pills per year per person Total pills per year Total pills per year times five| 1 | 157 | 7.175202 | 2798329 | 13991644 |
| 2 | 314 | 14.350404 | 5596658 | 27983288 |
| 3 | 471 | 21.525606 | 8394987 | 41974933 |
| 4 | 628 | 28.700809 | 11193315 | 55966577 |
| 5 | 785 | 35.876011 | 13991644 | 69958221 |
| 6 | 942 | 43.051213 | 16789973 | 83949865 |
| 7 | 1099 | 50.226415 | 19588302 | 97941509 |
| 8 | 1256 | 57.401617 | 22386631 | 111933154 |
| 9 | 1413 | 64.576819 | 25184960 | 125924798 |
| 10 | 1570 | 71.752022 | 27983288 | 139916442 |
If somebody takes 10 pills, this equals 1570 milligrams, or 1.6 grams, of MDMA in one occasion. This is not realistic; we would see many more incidents if people would take this much. But even with this gross overestimate, we would only total 27983288, or 28 million, pills per year in the Netherlands.
In Party Panel 15.1 we also asked what dose participants consider a high dose (both in pills and in milligrams), and what their preferred dose is (again both in pills and in milligrams). Participants indicated that they considered 2.37 pills a high dose, and preferred 1.57 pills. In grams, they indicated that they considered 208 milligrams of MDMA a high dose, and preferred 166 milligrams of MDMA.
This confirms that the assumption that people take 10 pills of 157 milligram of MDMA per occassion is not realistic. An estimate of 3 or 4 pills per occassion seems more plausible (but even then, on the high side). We’d then estimate the total pill consumption in the Netherlands at 11 193 315 or 11 million pills per year.
That is a big difference with the estimate of 194 313 376 from the report; to be precise, it is 5.76% of this estimate. Hardly a negligible difference: even if the error would turn out to be that in the report, a decimal point was misplaced, the estimate would still be unrealistic. Therefore, something must have gone fundamentally wrong with those calculations.
By the way, one other important comment about chapter 12. In table 12.3 the ‘ever used prevalences’ are shown. For ecstasy, this was 2.5% in 1997 and 7.6% in 2017. This is increase is cited as evidence for increase in use of ecstasy. However, this reasoning does not hold: given that ecstasy use is a relatively recent phenomenon, and that ‘ever used prevalence’ can never decrease (once you used ecstasy once, this will count for the rest of your life), it makes sense that the ‘ever used prevalence’ increased between 1997 and 2016, even if the percentage of people who uses ecstasy stays exactly the same (it could even decrease, and the ‘ever used prevalence’ would still show an increase). After all, people who were born in the 1930s only had the opportunity to use ecstasy in their fifties. The likelihood that they were still active nightlife participants is negligible, but they did count in the ‘ever prevalence’ in 1997 (they were 67 years old then). Twenty years later, in 2016, a part of this non-using group will have deceased. People who were twenty years younger were in their thirties when ecstasy was introduced in the Netherlands; of this group, a larger proportion will still have had the opportunity to use ecstasy, and a much smaller proportion will have passed away in 2016. The upshot is that an increase in the ‘ever used prevalence’ is a necessary consequence of the mechanics of ‘ever used prevalence’ in combination with the relatively late introduction of ecstasy in the Netherlands, and therefore, can not tell us anything about whether the prevalence of ecstasy use increased between 1997 and 2016 or not.
Where did the wrong numbers come from?
How is it possible that the numbers of consumed ecstasy pills were overestimated so grossly in this report?
Perhaps the estimates for the total production were correct, but almost all production is exported. However, this is inconsistent with the low quality of the ecstasy pills available in the countries around us.
Maybe the production is less efficient than the authors assumed, therefore their calculations on the basis of precursors were off, and the production is much, much lower than they estimated.
Or maybe another parameter in their calculations was wrong. They indicated that they used conservative estimates; that means that their impression of what was a conservative estimate was perhaps wrong.
Be that as it may, it remains very strange that no reality-check occurred at any point. 200 million is such a huge estimate that you don’t have to be an ecstasy researcher to realise that this is impossible.
Now, estimation is always very, very hard: and to err is human. I hope that the authors can use the bandwidth of realistic ecstasy consumption estimates as calculated here to find the mistakes in their calculations, so they can correct those.
In the meantime, the report does make clear that the present substance policies, in their present intensity, but especially in their present nature, fail. It is time to take step back and evaluate.