Guest Commentary: An industrywide call for greater antibody information

Few people have battled antibody unreliability to the extent of Steven Elliott, formerly one of Amgen’s scientific executive directors.

 

In 2001, Amgen received approval for an advanced erythropoietin-stimulating agent (ESA). As the drug became widely available, Elliott and researchers worldwide began to investigate whether the erythropoietin receptors (Epo-R) it targeted were widely expressed. After many years spent studying the receptor, Elliott was convinced they were only found in significant levels on erythroid progenitor cells.

 

However, other researchers arrived at a different conclusion. The problem was the antibodies they used to survey different cell types for the receptor also bound to off-target proteins, generating a signal that the scientists wrongly identified as Epo-R. Elliott, having identified the antibody issue, spent years talking and writing to these scientists to explain what was happening, but few listened.

 

Recalling how scientific literature and preclinical and clinical trials became contaminated with the incorrect notion that Epo-R is widely expressed, Elliott says, “All of it started because of these flawed antibodies that were improperly validated. It just spun out of control until it became the prevailing opinion.”

 

In reality, the scientists were using an antibody that detected a common heat shock protein (HSP70) and ascribed that band on the western blot to Epo-R. Researchers were building off the data supplied in peer-reviewed papers, with the expectation that if the reagents had been published, they were proven and validated for that use. This particular antibody has been used in 38 publications, according to CiteAb. However, all those studies failed to include a negative control cell type to detect false-positive data and used improper positive controls. The true Epo-R band showed up as a faint line below the HSP70 band—often cropped out of the submitted image.

 

The anti-Epo-R antibody is still marketed and used today.

 

 

Many powerful commentary pieces in the last few years have highlighted unreliability and irreproducibility at every level of science research, adding up to an estimated $28 billion wasted each year in irreproducible preclinical research in the United States alone. In 2014, the director of the National Institutes of Health (NIH), Francis Collins, called for action in a Nature article.

 

“The recent evidence showing the irreproducibility of significant numbers of biomedical-research publications demands immediate and substantive action,” wrote Collins.

 

Arguably the greatest need for change lies in the field of commercial antibodies. As the case study above shows, endemic quality-control issues—including non-specific binding—are propagated throughout academia, facilitated by a lack of supplementary data and scientific rigor. In 2012, C. Glenn Begley wrote a landmark paper that analyzed 53 preclinical studies that relied heavily on antibodies. His team found they could replicate only six of them.

 

Collins cites many complex reasons for the status quo, including a lack of researcher training and an academic system that incentivizes provocative statements. Another chief concern of his is the lack of data transparency demanded by peer-reviewed publications. Due to the minimal information required by journals, it can be hard to determine if studies, such as those using antibodies, involved the necessary controls.

 

Industry leaders are calling for change. Though proposed solutions differ considerably, a common theme has emerged: the need for greater transparency, from data in peer-reviewed journals to antibody vendors to the antibodies themselves.

 

 

Buying antibodies is not an easy task, and one that frequently lacks sufficient scientific rigor. Busy researchers often have little time to select antibodies for a study, so many follow precedent, selecting products colleagues have recommended or ones cited in a peer-reviewed journal.

 

To help streamline the process, several third-party websites are gaining traction as independent databases for antibody citations. For a researcher purchasing perhaps a dozen different antibodies for an experiment, having a central source for this information can save significant time.

 

CiteAb is a popular example, with more than 1.8 million antibodies listed and 15,000 unique visitors per month. The website was founded by Dr. Andrew Chalmers on the back of his own frustrations dealing with unreliable antibodies. Along with collating and ranking antibodies by number of citations, Chalmers says the goal is to publish more comprehensive antibody data.

 

To ensure that information is available for CiteAb to post, Chalmers says journals need to begin demanding many more details about the type of antibody used, including the name of the antibody provider, the catalog and batch number and what in-house validation the researchers performed.

 

He’s not alone in his views. As Collins noted in his Nature article: “Journals should be encouraged to devote more space to research conducted in an exemplary manner that reports negative findings, and should make room for papers that correct earlier work.”

 

With the advent of online peer-reviewed publications, it is now possible to include this information as supplementary data with any submitted paper. However, even if this avenue is maximized, the practice of relying on citations will continue to have limitations. The Epo-R case study underscores this point. As a result, Chalmers believes researchers always need to screen the reagents themselves and, where possible, use positive and negative controls.

 

“Once you’ve chosen an antibody, it’s very important to validate it and check that it’s behaving as you expect in your own lab. There is no getting away from that,” he says.

 

Authors also could help give context to their findings by providing details of their validation studies. Doing so would address, at least partially, the problem of publication bias.

 

Then, readers might understand that when a picture-perfect western blot is published, it doesn’t mean the study and the antibody are of the highest quality. It means the best blot was chosen, and potentially cropped, to highlight the bands of interest.

 

 

Some degree of change is surely in the antibody pipeline. What can vendors do in the interim?

 

The first step toward restoring scientists’ trust in commercial antibodies is weeding out products that simply don’t perform. Only antibodies that detect endogenous proteins should be sold for studies using direct immunodetection methodology (i.e., without prior enrichment using techniques such as immunoprecipitation pull-down). If an antibody has been screened on overexpressed protein samples, that should be made abundantly clear—it might mean the reagent is not suitable for direct detection of the target in naturally occurring sample concentrations.

 

For some targets, enrichment may be required prior to immunodetection. In all cases, vendors need to be clear what kind of proof of performance is being shown. Vendors should share the complete data generated with this antibody (for example, the full western blot when testing for westerns —not simply the lane or segment of the membrane that shows the positive band). Just because the signal was weak or missing in a certain sample doesn’t mean the information is not valuable. Rather, it may offer valuable clues as to how the reagent performs, increasing users’ chances of success.

 

Alongside the push for more information, researchers need to recognize antibodies’ inherent limitations. Science needs to move beyond the idea that antibodies can be given a blanket “pass” or “fail” grade as they’re streamed through a vendor’s quality-control process.

 

 

In a recent Nature commentary piece, Andrew Bradbury, a Los Alamos National Laboratory scientist, and Andreas Plückthun of the University of Zurich called for radical action far beyond greater product clarity. They believe that, given the severity of the waste and the damage to science, the antibody industry must be overhauled in two key ways. First, vendors must move from manufacturing and selling polyclonal antibodies to recombinant antibodies. Second, for all recombinant antibodies entering the market place, vendors should also make available the specifics of the product’s gene sequence. This gene-based paradigm, Bradbury says, is the only way to ensure antibodies from different companies are recognizing the same target and binding to the same site. The scientists also believe accompanying validation data should be lot-specific; vendors should not use historical data in data sheets as they may not relate to the batch being sold.

 

“If everybody is using the same antibody, you can be sure results can be directly comparable,” he says.

 

The move would also counter the familiar issue of lot-to-lot variability. With the status quo, the potential to accurately replicate an antibody study dies with the rabbit that generated those polyclonals.

 

However, Bradbury’s proposed actions face many hurdles, perhaps the strongest being the industry’s reliance on proprietary sequences to fund research. Individual companies are reluctant to release antibody sequence information that they perceive to be vital to their competitive advantage.

 

Additional issues to consider are that changes to product performance may occur due to differences in the manufacturing process that can affect activity, purity and stability. This means that a recombinant antibody produced by several different companies may not perform identically, even though the same DNA sequence was used. That said, there is little doubt this would lead to far greater reproducibility than the present use of animal-derived antibodies.

 

 

Multiple parties are doing their part to improve things, including Bio-Rad Laboratories itself, where we are attempting to deliver a product that better meets researchers’ needs. After acquiring AbDSerotec in late 2013, our team has been undertaking market research, listening to researchers’ pain points and gathering feedback from experts in this space—also, combining the antibody experience in AbDSerotec and the western blotting expertise in Bio-Rad to launch the PrecisionAb antibody product line this June.

 

The need for industry-wide changes in the way antibodies are manufactured, sold, validated, used and reported is now well established. As awareness of antibody unreliability and its consequences grows, many groups are lobbying to find a better way.

 

“This has become an epidemic,” Elliott says about the propagation of flawed scientific work. “In some cases the experiments are trivial or irrelevant, but in other cases it’s very troubling because there are antibody data that are published around important biological questions and if the antibody is wrong, or non-specific, or the data is misinterpreted, then it creates a huge impact on the scientific community.”

 

While a total shift to recombinant proteins may not yet be feasible, transparency is achievable today. Antibody users should be fully informed—the integrity of science depends on it.

---

Roumen Bogoev is the antibody content manager at Bio-Rad Laboratories. Read more about the PrecisionAb antibodies at the company’s website.