Harmonisation of multi-centre real-time reverse-transcribed PCR results of a candidate prognostic marker in breast cancer: An EU-FP6 supported study of members of the EORTC – PathoBiology Group

1. Introduction 

The measurement of biomarkers in tumour tissue is finding increasing use in determining prognosis and therapy prediction in patients with cancer. Two of the earliest biomarkers in breast cancer were the steroid hormone receptors: estrogen receptor and progesterone receptor (ER and PgR). As the degree of benefit from endocrine therapy is directly related to the quantity of receptor present in the tumour1, the quality of the steroid receptor assays is important. Moreover, since patients entering in multi-centre trials are often stratified based on the receptor status, data obtained with steroid hormone receptor assays should be comparable between different institutes. Our laboratory has performed ER and PgR quality assessment studies for almost 20 years for the EORTC Receptor and Biomarker Study Group, now the PathoBiology Group.2, 3 This has resulted in a significant increase in precision of the ER and PgR assays applied by the participating laboratories for the assessment of hormone receptor status for breast cancer patient treatment selection. Similarly, external quality assurance (EQA) has been provided for the measurement of urokinase-type plasminogen activator (uPA) and plasminogen activator inhibitor type 1 (PAI-1) in breast cancer tissue.4 Both uPA and PAI-1 have reached Level-of-Evidence (LOE) I5, 6, 7 as prognostic biomarkers for breast cancer, having undergone validation using both a randomised prospective trial8 and a large pooled analysis study.9 The EQA for both these studies was carried out in our department.4 The department of Chemical Endocrinology is also responsible for EQA in the Node-Negative Breast Cancer (NNBC)-3-Europe trial, where participating centres perform risk estimation for breast cancer patients by measuring the uPA and PAI-1 concentration in tumour tissue biopsies.

Due to its sensitivity (requiring only a minute amount of tissue), its accuracy and its quantitative characteristics, qRT-PCR is increasingly being used to determine the prognostic value of potential biomarkers in tumour tissues. Not surprisingly, most of the recently identified predictive and prognostic biomarkers are mRNA-based. As part of the validation of prognostic and predictive biomarkers, multi-centre studies enhance the level of evidence of a potential biomarker.6 Thus, for qRT-PCR of particular biomarkers to achieve a high level of evidence, EQA programmes are required that aid in harmonising results obtained in different centres.

We previously showed in ER-positive breast cancer that DNA methylation of the paired-like homeodomain transcription factor 2 (PITX2) gene was associated with poor outcome in lymph node (LN) negative breast cancer patients whether having received adjuvant endocrine therapy or no adjuvant systemic therapy10, 11, and in LN positive patients who received adjuvant anthracyclin-based chemotherapy (Hartmann et al., submitted for publication). PITX2 is a WNT and/or TGF-beta family driven differentiation marker that contributes importantly to the pituitary gland development12, including terminal differentiation of the somatotroph and lactotroph lineages.13 PITX2 is also essential for proper left-right patterning.14 Three main isoforms PITX2A, -B and –C, which are translated from 3 different mRNA splice variants transcribed from the gene, have been described. Recent results have indicated that PITX2 mRNA expression next to PITX2 DNA methylation is a prognostic marker for breast cancer (Martens et al., in preparation).

The above-mentioned results were obtained within an EORTC-PathoBiology Group, EU funded FP6 study ‘DNA Methylation’. In the current study, 3 rounds of EQA were used to quantify and if possible reduce the inter- and intra-assay variability of qRT-PCR in and between 5 laboratories from 4 European countries that participated in this FP6 study. We quantified by qRT-PCR the potential biomarker PITX2 and its splice variants, and three reference genes, B2M, HPRT1 and HMBS, in externally provided biological samples. The main objective of the first EQA round was to monitor adherence to established consensus protocols and consistency in assay methodologies, and to determine base-line inter- and intra-assay variability. The second EQA round was used to obtain data to investigate the beneficial effect of the use of calibrators. In the last round, the effect of normalisation of the results was studied using internal reference genes and by centrally harmonising the qRT-PCR results.

2. Material and methods 

2.2. RNA extraction 

For the calibrators, RNA extraction from the cell lines was performed by the central QA laboratory. Total RNA was isolated using the RNeasy mini kit (Qiagen, Hilden, Germany) with on-column DNase-I treatment. Quality of the RNA was checked by examining ribosomal RNA bands after agarose gel electrophoresis. RNA concentrations were determined from the spectrophotometric absorption at 260nm using the Genequant (Amersham, Eindhoven, The Netherlands). Similarities and dissimilarities between the RNA extraction methods used by the participants in this study are listed in Table 1.

Table 1. Methods of participants.

		Participant	1	2	3	4	5
	RNA extraction	Method
		Trizol				X
		RNABee	X
		RNeasy		X	X		X
		DNase		X		X	X
	RT reaction	Enzyme
		M-MLV			X		X
		Superscript	X	X		X
		Primers
		Random hexamers	X	X	X	X	X
		Oligo-dT	X				X
		RNAse-H treatment	X				X
	PCR reaction	Apparatus
		AbiPrism	X	X	X	X
		Mx3000P	X
		LightCycler					X
		Mix
		Stratagene	X
		Applied Biosystems	X	X	X	X
		Qiagen					X
		Protocol	Denaturation,	Denaturation,	Denaturation,	Denaturation,	Denaturation,
			35 cycles:	40 cycles:	40 cycles:	50 cycles:	35 cycles:
			15s 95°C	15s 95°C	15s 95°C	15s 95°C	15s 95°C
			30s 62°C	60s 60°C	60s 60°C	60s 65°C	30s 60°C
			20s 72°C		Melting		30s 72°C
			20s 79°C		curve		Melting
			Melting curve	Melting curve			curve

3. Results 

3.1. First EQA round 

3.1.1. Inventory of consensus protocols, procedures and equipment of participants 

To determine sources of variation among participants a questionnaire on the procedures, equipment and protocols used by the participants was sent out. These participants were from Rotterdam (Department of Medical Oncology, Erasmus Medical Center Rotterdam, The Netherlands), Dublin (Department of Pathology and Laboratory Medicine, St. Vincent’s University Hospital, Dublin, Ireland), St Cloud (Laboratoire d’Oncogénétique, Centre René Huguenin, St-Cloud, France), Basel (Stiftung Tumorbank Basel, Riehen, Switzerland) and Nijmegen (Department of Chemical Endocrinology, Radboud University Nijmegen Medical Centre, Nijmegen, The Netherlands).

The feedback on this questionnaire is summarised in Table 1. There were significant differences in the standard protocols between the laboratories. RNA was extracted using Trizol, RNABee or RNeasy mini columns, and RNA quality control was performed by agarose gel electrophoresis (3 of 5 participants) or Agilent Bioanalyzer (2/5). The RT reaction was done using M-MLV (2/5) or superscript II RNase H-reverse transcriptase kit from Invitrogen (3/5). All participants used random hexamers, whereas 2 laboratories also included oligo-dT primers. Two of the participants applied an RNAse-H treatment to the obtained cDNA. All participants had at least yearly PCR apparatus calibration and maintenance scheme. Four of the 5 participants used the Applied-Biosystems Abi Prism (7000/7700) PCR apparatus, 1 participant used in addition the Mx3000P Real-Time PCR System from Stratagene, whereas 1 participant used the LightCycler from Roche.

3.1.2. Establishment of inter- and intra-laboratory variation among participants 

To determine the inter- and intra-laboratory variation, EQA samples were prepared by the central QA laboratory and distributed to all partners. The samples consisted of 4 vials of total RNA, isolated from different cell lines (MDA-MB175VII, OVCAR-3, MCF7 and JEG-3) and 4 vials of cDNA, prepared from the RNA mentioned above by the central QA laboratory. In addition, one no-template control (NTC) sample was sent. The samples were sent on dry-ice to ensure integrity. Reagents necessary to perform the RT and PCR reactions for this EQA round were also distributed to the appropriate partners. This was done to establish whether differences in the data obtained between the partners could be attributed to the differences in reagents. For this first EQA round, primers for real-time PCR for 3 reference genes (B2M, HPRT1 and HMBS) were distributed among the participants.

In this first QA round, the intra-laboratory CV was 3–26 (mean approximately 10%, Fig. 1A), while the inter-laboratory CV was 36–70% (Fig. 1B). This large inter-laboratory CV was not explained by the RT-reaction as it was similar when the participants performed the RT reaction themselves (using the RNA that was received from the central laboratory) or when externally provided cDNA was used (Fig. 1B).

View Large Image Download to PowerPoint

Fig. 1. First EQA round. (A) Intra-laboratory CV for each participant per assay. (B) Inter-laboratory CV of samples that were reverse-transcribed by the participants according to their in-house RT protocol or by the central EQA laboratory.

3.2. Second EQA round 

Participants received prediluted calibrators for PITX2 and its three splice variants and 5 unknowns to be measured in triplicate and quantified using the calibrators. The calibrators consisted of dilutions of cDNA from cell lines that highly expressed the gene(s) of interest. The average intra-laboratory CVs for all 5 samples are shown in Fig. 2A for each separate assay, whereas Fig. 2B shows the average intra-laboratory CVs for all samples and all assays. Clearly, laboratories 1, 2 and 5 showed acceptable variations (<15%), whereas laboratories 3 and 4 had higher CVs ranging from 12 to 30%, depending on the assay. The average intra-laboratory CV ranged from 7 to 21% (Fig. 2B). For the PITX2, PITX2A and PITX2C assays, the inter-laboratory variation ranged from 19 to 24% (Fig. 2C), with the variation for PITX2B assay being as high as 53%. This was caused by the consistently higher values for PITX2B reported by participant 5. Excluding the results from participant 5, the inter-laboratory CV for the PITX2B assay was 28% (not shown).

View Large Image Download to PowerPoint

Fig. 2. Second EQA round: intra-laboratory CV for each participant, either per assay (A) or on average for all assays (B). 1–5 denote the different participants. (C) Inter-laboratory CVs per assay.

3.3. Third EQA round 

For the final round, both prediluted calibrator curves and cDNAs from 8 unknown samples (obtained from SW480, HEPG2, JEG3, JAR, MCF7 (2x), MDA-MB175VII and OVCAR3 cell lines) were sent to all participants. The average intra-laboratory CV for the final EQA round for all samples are shown in Fig. 3A, for each separate assay, whereas Fig. 3B shows the average intra-laboratory CV for all samples and all assays in this EQA round. For this analysis, samples with levels <10 (AU) were considered negative. The intra-laboratory CV ranged from 7 to 23% depending on the participant. The inter-laboratory CV ranged from 15 to 29%, with a mean inter-laboratory CV of 20% (Fig. 3C). Subsequently, all data were normalised with a harmonisation procedure. For this, the data from a participant were corrected using the ratio of the average of all the data from a participant over the average of all the data from all the participants. If a particular participant reported consistently higher (or lower) results, this procedure corrected for that. The harmonisation factors per assay and per participant are shown in Fig. 4A. Especially for PITX2B, the differences between laboratories were high (nearly 30%). For the other assays, the inter-laboratory CV was 15–23%. Some participants exhibited consistently lower values (as exemplified by a harmonisation factor >1, e.g. for partner 3), or higher values (e.g. partner 5), requiring a harmonisation factor <1 (Fig. 4A).

View Large Image Download to PowerPoint

Fig. 3. Third EQA round: intra-laboratory CV for each participant, either per assay (A) or on average for all assays (B). 1–5 denote the different participants. (C) Inter-laboratory CVs per assay. PITX2A and B2M values for participant 4 are missing.

View Large Image Download to PowerPoint

Fig. 4. (A) Harmonisation factors for the different assays and participants. (B) Effect of harmonisation of data on inter-laboratory CV per assay. (C) Effect of normalisation – using reference gene expression – of data on inter-laboratory CV per assay.

As a result, the inter-laboratory CV of assays for which such consistent deviations occurred was diminished by harmonisation. As can be seen in Fig. 4B, the inter-laboratory CV of PITX2B and B2M was significantly attenuated by harmonisation. For PITX2A there was no effect, whereas the other assays showed a minor improvement in the inter-laboratory CV. Normalisation of the data using reference gene expression alone had no significant effect on the between-laboratory variation (Fig. 4C). In conclusion, while normalisation of the data using reference gene expression was ineffective our harmonisation procedure reduced the average between-laboratory variation from 25% (range 15–29%) to an acceptable 13% (range 10–17%).

4. Discussion 

One of the aims of our 6th FP: ‘DNA-methylation’ project was to quantify at the mRNA level multiple potential prognostic and predictive epigenetic markers in breast cancer. One of the mRNA species investigated was PITX2, the epigenetic modulation of which was shown to be a strong prognostic marker10, 11 (Hartmann et al., submitted for publication). Prior to the various participating centres embarking on the measurement of this as well as other mRNA transcripts, we carried out EQA on the qRT-PCR procedure for quantifying PITX2 and its variants. Our attempts to harmonise the results obtained in the different centres are described in this report. Intra-laboratory CVs ranged between 5 and 23%, and the inter-laboratory CV was 17–30%. Differences between laboratories could not be normalised by correcting for reference gene expression. However, harmonisation of data further attenuated the between-laboratory variation to 13%, which seems excellent given the use of different protocols and PCR platforms.

The robustness of the results obtained in qRT-PCR of RNA obtained from tumour tissue samples largely depends on the quality of the starting material used. Therefore, before the harmonisation of results experiments started, all participants of the 6th FP: ‘DNA-methylation’ were trained in tissue handling, and DNA and RNA isolation. Uniformity of protocols was ascertained during training of technical laboratory personnel of each of the participating institutes at the bench in the quality assurance laboratory at the start of the project (Workshop Nijmegen, ‘Nucleic Acid extraction and Quantification’, May 17–18, 2004). At this workshop, consensus protocols for tissue, RNA and cDNA handling, quality control, and real-time PCR performance and data handling were developed and distributed among the participants. Most importantly, crucial steps in the RNA and DNA handling procedures were identified. Recommendations were made to use tissue sections for morphological examination and to establish the percentage tumour cells, on the procedure used to extract DNA from the samples, and to use column-based RNA extraction procedure if possible. Furthermore, spectrophotometry and gel electrophoreses, or if possible a Bioanalyser, should be used to assess RNA quality. DNAse treatment is considered an option only if the PCR is sensitive for genomic DNA and the primers cannot be changed. On column DNAse treatment would then be preferable. The best choice of house-keeper(s) is at least HPRT and PBGD, and a third housekeeper depending on tissue. The participants were made aware that several methods for normalisation of data exist. When using housekeepers, absolute quantification (using e.g. plasmid constructs) of both the gene of interest and the housekeeper is best. Finally, the so-called Delta-Ct method can only be used when the target and housekeepers are amplified with the same efficiency.

Previously, an Italian inter-laboratory comparison from 42 laboratories16, 17 identified high variability in performance among laboratories using TaqMan probes in conjunction with a variety of real-time PCR platforms. Similarly, EQUAL-quant is a European Union funded multidisciplinary EQA scheme that included 137 participating laboratories from 29 countries.18 Both these extensive EQA programmes showed that results from some participating laboratories fell well out of the range of the other partners. The conclusion was that these EQA schemes demonstrated both the requirement and demand for external assessment of technical standards in real-time PCR. The data obtained during our EQA scheme, although only encompassing 5 laboratories, were encouraging. Despite differences in analysis methodology and PCR equipment, the inter-laboratory CV could be reduced to approximately 13% using prediluted calibrators and by harmonising the data in a central QA laboratory. Given the high sensitivity and dynamic range of real-time RT-PCR, we feel that these variations are low. For ER protein measurements, inter-laboratory CVs of approximately 30% (using Enzyme-Immuno Assays) to 40% (using Ligand Binding Assays) were found2, which could be attenuated to 10–15% by harmonisation.3 Similarly, with ELISAs for uPA and PAI-1 the inter-laboratory CVs of 40–60% were found that could be reduced to 15–20% using common external standards sent out by the central QA laboratory.4

In conclusion, for multi-centre validation studies of biomarkers quantified by qRT-PCR, it is necessary to use externally provided calibrators for the genes of interest to obtain comparable results between different laboratories. Subsequent harmonisation will further reduce variation in qRT-PCR results, and both are required to reduce the variation to an acceptable level for multi-centre measurement of prognostic and predictive ribonucleotides in breast cancer. With many RNA-based prognostic markers being revealed by us (e.g.19, 20, 21) and others in the current omics-era, EQA programmes as the encouraging one performed here will be needed to lift mRNA-based markers to a higher level of evidence needed for them to initiate prospective validation, which is required for actual clinical application. Based on the current promising results and preliminary data that PITX2 at the mRNA levels is a prognostic marker (manuscript in preparation), the involved laboratories have initiated a multi-centre validation of PITX2 mRNA measurement as a potential new prognostic marker in breast cancer.

Conflict of interest statement 

None declared.