NOOTROPICS SURVEY RESULTS AND ANALYSIS

 http://slatestarcodex.com/2014/02/16/nootropics-survey-results-and-analysis/

POSTED ON FEBRUARY 16, 2014 BY SCOTT ALEXANDER

In case you are just joining us: nootropics are substances which purportedly improve mental functioning with relatively few side effects when used responsibility. 

Caffeine is a good example. It improves wakefulness and energy, and when used responsibly is so safe it can be sold “over the counter” in coffee and other drinks.

Over the past few years, various substances from experimental drugs to common supplements have been claimed to satisfy these criteria. If they really work, obviously we want to know about it.

Double-blind randomized controlled trials have confirmed exciting properties for some purported nootropics. A meta-analysis has confirmed that the neuroprotective drug piracetam improves cognition in demented patients, and smaller studies suggest it does the same in healthy individuals. A meta-analysis of traditional Ayurvedic medicine bacopa monnieri finds it improves memory and reaction time in healthy subjects.

The online nootropics community consists of mostly healthy individuals looking to use these substances to gain motivation or clearer thinking. They tend to “outpace” the scientific literature, using some substances that haven’t yet been shown to work, or haven’t been shown to benefit healthy individuals. Information travels mostly through word of mouth, and people combine a bewildering variety of chemicals without being sure which ones work better than others or even whether any of them work at all.

This survey does not intend to rectify this state of affairs, but it can at least shed some light on it.

I asked people in the two largest online nootropics communities (that I know of), Reddit’s r/nootropics and Longecity’s Brain Health forum, to take a survey describing their use of 31 different substances. I got 162 responses. My respondents were 92% male and 8% female, with an average age of 25. Most of them had tried only a few different chemicals, but a few had sampled nearly two dozen. Most were taking “stacks” of several different chemicals daily.

Three substances – DMAE, centrophenoxine, and a combination stack called TULIP – got fewer than ten responses, so I removed them from the survey because there wasn’t enough of a sample size to do anything with.

Below I describe the results I got on the remaining 28 substances and some hypotheses that I tested:

Head-to-Head Comparison Of Nootropics

Respondents were asked to rate their subjective experiences of different nootropics on a scale of zero (completely useless, did nothing) to ten (life-changing). Comments on Reddit suggest several people misunderstood the scale, but not much I can do about that at this point.

I asked people to reserve numbers five or above for effects that, in their subjective opinion, couldn’t possibly be placebo – but the whole point of the placebo effect is that people can’t consistently make this judgment. I would have liked to have some placebo nootropics to use as a comparison, but since I wasn’t running a trial, I didn’t have that option.

I did, however, try to assess the placebo effect in a couple of sneaky ways. One of the nootropics measured, choline bitartrate, has carefully regulated ability to cross the blood-brain barrier beyond a certain level, and so it would be surprising if it had direct and immediate cognitive effects. Nevertheless, 27 respondents had used this substance and they gave it an average rating of 3.7 out of ten. This suggests that 3.7 out of ten is a good first guess for the placebo effect.

Another of the nootropics measured, bacopa monnieri, has been found in multiple studies not to exert any effect before a month of consistent use. 6 respondents had used this substance for 20 days or less, and they gave it an average rating of 4 – which is very close to 3.7 and so confirms our previous guess.

These methods aren’t perfect – there are some mechanisms by which choline can relieve deficiency even in the absence of direct cross-BBB transport, and some people say that at least some anxiolytic effects of bacopa are immediate – but they’re the best we’ve got. So let’s say anything that comes up 4 or less has a good chance of being placebo. 

This does not imply that numbers above 4 are therefore necessarily real effects; we know that more promising drugs exert stronger placebo effects. Thus, in the early days of antidepressants when no one believed they would work, they had a very weak placebo effect, whereas after the culture was saturated with messages of antidepressant efficacy, they started having a much stronger placebo effect. So a much-vaunted nootropic might well be able to muster a placebo effect somewhat greater than 4.

Below, I give tables ranking the nootropics studied along several axes.

The mean and median ratings for each substance

The percent of respondents who rated each substance as >=5 (ie they could not believe it was placebo), and = 10 (ie life-changing)

The number of respondents (out of 162) who used each substance, and the mean number of times they have used it.

I did not include standard deviations in these tables, so they conceal the true amount of variation in each substance. In general, nearly every substance had at least one zero and at least one nine or ten. Very few showed an obvious consensus on whether they worked or not. I reproduce below a graph of piracetam – one of the most-used nootropics – as a very typical example:

The results themselves were not surprising. The most effective nootropics were caffeine – which pretty much everyone in the world knows works – and modafinil (along with its fellow -afinils) which has been proven to work by countless studies and is a prescription drug much used by the medical establishment.

Below these two, the most effective “exotic” nootropics were phenylpiracetam and coluracetam. I suspect that despite the nootropic community’s enthusiasm about phenylpiracetam it mostly just works as a mild stimulant not too different from other stimulants like amphetamines. Coluracetam, a very new and pretty unique substance currently being investigated as an Alzheimer’s treatment, is more exciting.

Of the next two, Noopept is a Russian prescription drug mostly unknown in the United States; Russian studies suggest it improves memory and learning. Theanine is one of the active ingredients in tea and is widely considered calming. Their success should not surprise anybody paying attention.

Creatine is a chemical related to amino acids. Although it seemed to do well, most people who ranked it highly mentioned in the comments that they liked it as an exercise supplement but didn’t get any cognitive benefits from it. It should probably be disqualified.

Ayurvedic medicines had a decidedly blah performance. A lot of people swear by rhodiola, ashwagandha, and bacopa, and they all seem to have a moderate amount of experimental support. But they were firmly in the middle of the pack here, outperforming our 4 threshold for placebo but only barely. Much like piracetam above, these were all over the place: here is the graph of rhodiola, the best-performing of the bunch:

An optimist might say that everyone has a unique biochemistry. A pessimist might say that everyone has a unique ability to extract patterns from random noise. I leave the choice between these two hypotheses to the reader.

Two other traditional Eastern medications – ginseng and gingko – fell completely flat, with both doing (nonsignificantly) worse than our suspected placebo level.

Dosing Issues

A secondary goal of this survey was to inspect the dose-response relationship of these substances and see whether the recommended doses were really set at the most effective level. My original plan was to report the dose at which a substance was most effective, but this proved beyond my technical abilities. Instead I report the median dose given by all users of a substance:

In milligrams, except where it isn’t. Vitamin D is IUs.

I investigated dosing of two substances in more detail – piracetam and vitamin D – because I had good data on both of them and there has been some controversy on correct dosing.

The median dose of piracetam was 1600 mg. I ran a t-test to see whether people who took high doses (>1600) had stronger effects than those who took low doses (< 1600) - the means looked different (5.6 vs. 4.4) but the effect was only marginally significant (p = .09). I tried to see if I could break the significance barrier with a larger dichotomy and so compared people taking below the 25th percentile (1200) to those taking above the 75th percentile (3000). This actually had less difference – means of 4.42 vs. 4.88 – so I gave up. An attempt to correlate dose to response with Pearson’s r also returned uninspiring results of r = -.04, p = .75. 

The whole analysis doesn’t really give any good reason to use anything other than the median dose of 1600 mg.

Vitamin D was much more interesting. I compared people taking more than the Vitamin D mean dose of 3000 IU with those taking less – the means seemed to differ (5.0 vs. 3.9), but the significance wasn’t there (p = .17).

Once again I tried to gun for significance by switching to quartiles – in this case, below 1625 IU vitamin D vs. above 5000. The means widened to 5.3 vs. 3.4, and the p value reached 0.8, within a hair-breadth of significance.

I decided to make a last desperate effort and restrict the analysis to the true extremes. I compared the people taking OVER NINE THOUSAND units to those taking 1000 or less. Here the means were 7.4 vs. 3.1, and p was 0.02. I am not entirely confident in this analysis since there were some small subgroups – n = 12 in the very low group and n = 5 in the very high.

Also, it should be kept in mind that these correlations might not be causal in the expected direction – just to raise one possibility, maybe people who really like vitamin D take more of it.

I investigated a bunch of other substances for dose-response relationships using bivariate correlations and couldn’t find anything. Overall my attempts to investigate dosing were a disappointment.

Correlations

My first several attempts to correlate different nootropics with each other were failures. Everything correlated with everything else due to a general “low standards factor” – some people were just very effusive and willing to give high ratings to everything, while other people had a much higher threshold before getting excited.

I did something very sketchy to adjust for this, and divided every person’s ratings for each substance by their average rating for all substances, so that each substance’s “rating” was transformed into the degree for which it was below or above average for that particular rater. This introduced a whole host of complicated biases, but it at least got me to the point where I could run correlations between substances without everything lighting up simultaneously.

By correlating 28 substances with one another I ran 378 separate correlations, so with a significance threshold of p = .005 I would expect about two hits by chance. Instead I got about a dozen.

Some of these were expected – for example, the -afinils correlated with each other. Others were bizarre and nonsensical. I have removed the inter-afinil correlations as too obvious to be worth noting, and am left with the following:

ALCAR -> Modafinil -.733, .000
Vitamin D -> Theanine +.591, .000
Creatine -> Noopept -.629, .001
Vitamin D -> Rhodiola -.634, .002
Piracetam -> Sulbutiamine +.683 .002
Piracetam -> ALCAR +.548 .003
Piracetam -> Modafinil -.505 .003
Phenylpiracetam -> Choline -.681 .004
Huperzine -> Creatine -.810, .005
Gingko -> Piracetam +.602 .005

Bizarrely, none of the racetams correlated with each other!

I have very low confidence that these represent real effects rather than artifacts of the gymnastics I had to do to make the data correlatable at all, and I report them only to encourage other people to do more sober analyses.

Odds and Ends

Choline is a commonly discussed nootropic used both on its own and as an adjunct to other substances. There are four main kinds of choline – lecithin, choline bitartrate, CDP-choline, and alpha-GPC choline. The first two do not effectively cross the blood-brain barrier beyond a certain level and so should theoretically be pretty ineffective for nootropic purposes. The second two do directly cross the blood-brain barrier and may have an effect. When I asked people to rate their choline, I got the following means:

Choline bitartrate: 3.7 (n = 27)
Lecithin: 3.8 (n = 5)
Alpha-GPC choline: 5.07 (n = 15)
CDP-choline: 5.37 (n = 19)

I ran a t-test comparing the two presumably ineffective forms of choline to the two potentially neuroactive ones and found a significant result (p = .045). This suggests that the forms of choline that could theoretically work are more effective than than ones that don’t – or that people are aware of the chemical properties of their forms of choline and adjusting their placebo effects accordingly.

Choline is sometimes used to improve the effects of -racetams, so I investigated whether people who took active forms of choline had better experiences with racetams than those who took inactive forms of choline. Out of six racetams, none of them achieved significance (although five out of the six had trends in the expected direction). This stayed pretty much the same when I restricted the analysis to people who told me specifically that they used choline with their racetams.

Choline is also sometimes used to prevent headaches with racetams, so I asked people about that. 40% said they found choline to help prevent headaches with racetams, 26% said they found it didn’t, and 34% just took choline because everyone else told them they should and didn’t know one way or the other.

The other category of drugs I looked at in more detail were the afinils. There are three common afinils. Modafinil is a well-known prescription medicine used by doctors as a wakefulness-promoting agent to treat narcolepsy, shift-work sleep disorder, and a couple other diseases. Adrafinil is an over-the-counter substance that gets metabolized to modafinil and should have about the same effecst. Armodafinil is an isomer of modafinil that should do about the same thing but require a lower dose.

I asked respondents if they had any preferences among these three similar substances. Of those willing to compare adrafinil and modafinil, 50% said modafinil was better, only 8% thought adrafinil was better, and 42% didn’t want to pick a favorite.

Of those willing to compare modafinil and armodafinil, 62% thought armodafinil was better, 23% thought modafinil was better, and 15% didn’t want to pick a favorite.

There is some theoretical concern about liver-related side effects from adrafinil, but some pharmacologists say these are overblown and that its liver profile is similar to Tylenol – ie don’t take a massive overdose on it or use it every day for years and you’ll be fine. None of my respondents reported ever having any liver problems with adrafinil – not even asymptomatic elevation of liver enzymes – but n = only 17 so the results don’t rule out even moderately common adverse reactions.

Last of all, there are some worries about tolerance to modafinil. Although the medical community is happy to give it to patients for several years straight and expects its results to remain just as strong as when it was first prescribed, the nootropics community has a lot of anecdotal evidence of tolerance developing if it is used too much.

I compared the modafinil experience ratings of people who used the substance occasionally (n = 52) to people who used it every day (n = 7). This corresponded, as expected, to a wide gap in how many doses they had taken on average – the occasional users had taken it 39 times, compared to the everyday users’ 184. However, the everyday users did not report decreased satisfaction with modafinil – both groups gave it an average satisfaction rating of 7.4. This suggests that at least this small subgroup of 7 people are not experiencing tolerance. We should, however, be careful to worry that they might be self-selected for exactly that quality – maybe a lot people try to use it everyday, but all but a rare group of non-tolerance-developers later back off.

Conclusions and Future Research

Overall I was not too impressed with the results of this survey. No unexpected substances jumped out as promising, there was a very wide distribution from “useless” to “life-changing” in nearly everything except the -afinils, and it was very hard to find much of a dose-response relationship anywhere. There is no “smoking gun” for most these substances being definitely-exciting, aside from the ones like modafinil that we were already aware of.

I repeat the other survey I run yearly and I’d like to establish a similar tradition here. Next year I hope this survey will have earned me enough cred in the nootropics community to be able to harass more people into taking it – whether that means getting it stickied on top of appropriate forums, getting nootropics distributors to advertise on their sites, or just getting more word of mouth. I know there are several thousand people in the community and I feel like I could do a lot more if I had two or three times as many responses.

Other goals for the future are to drag people kicking and screaming into reading directions so I know everyone’s rating off the same scale, to ask more specifically about tolerance development, and perhaps to separate main effects from side effects (although that brings up a lot of complications I will have to sort through).

Below are the raw data available for download, minus about a dozen people who asked for their entries to stay private. I am making it available in two forms. First are the raw raw data, which include people’s comments, people’s stacks, and a whole bunch of incorrectly entered machine-unreadable responses that I had to go through by hand and correct ಠ_ಠ . The sanitized raw data are clear, pure numbers that you can get to work on right away, if you don’t mind the fact that I deleted a lot of ambiguous responses and removed some stuff entirely just because keeping it seemed like a lot of work. I put a very brief explanation of how I coded some bits of the sanitized data up here; otherwise you’re on your own or can ask me questions if it’s too confusing.

I know Gwern is hoping to analyze these data further and I look forward to seeing what other people can make of them.