Suitability of frozen surgical pathology material
The adequacy of frozen surgical pathology material for microarray analyses can vary substantially. With the advent of linear RNA amplification protocols, the adequacy issue relates more to quality than quantity of material. Indeed, needle biopsy material can be used for microarray-based assays [24, 25] . It is RNA quality that is a critical determinant of the reliability of microarray-based expression profiling results. Conventionally handled material that is frozen at the time of gross examination of the specimen or at the time of frozen section if one was performed (and the piece was sufficient for procurement) may often show too much RNA degradation to provide an adequate sample for expression profiling (even though it is usually adequate for RT-PCR based analyses such as fusion transcript testing). The failure rates vary widely depending on the time intervals and type of tissue. Although the ability to obtain adequate RNA for microarray analysis in only 50% samples may be workable in a research setting, if the microarray data are needed for clinical reasons, this situation is not acceptable. Procurement and snap-freezing of tissue in the operating room can decrease failure rates to less than 10%. However, this may mean that a portion of the material bypasses gross and microscopic examination by a surgical pathologist, a situation with potential political and medicolegal implications. The pressure for intra-operative procurement and snap-freezing of tissue for microarray research studies is increasing and this trend is likely to accelerate as clinical tests using microarray-based expression profiling become established and accepted.

Inter-laboratory reproducibility of gene expression profiles
Major sources of poor reproducibility include different tissue handling and RNA extraction methods, different microarray platforms, different statistical analytic methods, different study designs (e.g different comparison groups), and the variability introduced by relatively small patient numbers. This variability is perhaps more notable in prognostic studies than in class prediction studies. For most tumor types, the issues of inter-laboratory and inter-platform reproducibility in the expression profiling of human tumors are only beginning to be addressed.

Using replicate hybridizations of a reference RNA sample that met strict quality-control criteria, Dumur et al. [22] found that showed that the Affymetrix Genechip system is highly reproducible, with the main source of the minimal variations being the day-to-day variability, and not variability between chips. Likewise, in a study of Affymetrix chips coordinated by multiple institutional microarray facilities, the major source of variability was between laboratories and not between chips (even from different chip lot numbers) [26] . This "operator-introduced" variability relates largely to subtle differences in RNA extraction methods and probe preparation and labeling techniques between labs. Performing replicates (e.g. triplicates) can substantially increase the reliability of gene expression data [27] but are not likely to be possible clinically.

Different microarray platforms pose a special challenge for reproducibility. For instance, in a comparison of genes most differentially expressed in pancreatic cancers as compared with nonneoplastic tissues as defined by either Affymetrix oligonucleotide chips or custom spotted cDNA microarrays, it was found that only 32 of 381 differentially expressed genes (8.3%) identified by these two platforms overlapped [28] . In general, it was noted that differentially expressed genes whose expression levels were high were more likely to be identified across different platforms. Other studies have also found (non-commercial) custom spotted cDNA microarrays to show poor concordance with Affymetrix chips [29] . There is only a slightly better concordance between different commercial products. A comparison of Affymetrix (oligonucleotide), Agilent (cDNA), and Amersham (oligonucleotide) chips found that while replicates on a given platform had excellent correlation coefficients (r>0.9), matched measurements across platforms had consistently poorer correlation coefficients (r=0.48 to 0.59), with no single manufacturer being distinctly superior [30] . Thus, at the present time, differences in results obtained on different types of chips or microarrays appear largely due to the arrays themselves. The flip side of this finding is that different platforms may still play a useful complementary role in the research setting.

Another possible source of poor reproducibility between different microarray platforms is the problem of incorrect annotations of individual elements (probes or cDNAs) on the microarrays. This may in rare cases reflect simple errors. However, in most cases, it is due to the uncertain identification of some genes at the time the array was designed. The human genome sequence, although largely complete, has still not reached the point where all exons are defined unambiguously. Thus, Mecham et al. [31] found that probes containing the most reliable sequence information showed higher reproducibility on the same platform (both on the same chips and different versions of chips), as well as higher reproducibility on the other platforms. This has also been observed by others [32] .

In terms of statistical analytic methods, microarray studies face a huge multiple testing problem which makes the standard p<0.05 cutoff unusable. Typically, smaller microarray studies employ methods to identify potentially biologically significant genes (for instance by using the False Discovery Rate method) that might not be judged statistically significant after a strict Bonferroni adjustment of the p value. This inevitably introduces "noise" in the differentially expressed gene lists. The variability introduced by relatively small patient numbers cannot be underestimated. As the field of expression profiling matures, it has become increasingly obvious that profiles based on small numbers of samples, e.g. less than 10 per group compared, should be viewed with great caution, regardless of their apparent significance as defined by often complex statistical methods.

Different study designs are also of obvious importance in comparing expression profiles across studies. It is self-evident that the profile of differentially expressed genes identified for a given tumor entity in a microarray-based study is partly dependent on the tumor types (or normal tissues) in the comparison group. However, the choice of comparison group is often dictated at least partly by convenience and does not always make biological sense (although interesting insights may still be gained). Few if any genes in any given tumor are expressed solely in that tumor, hence the impact of different comparison groups on the resulting lists of differentially expressed genes.

In the area of microarray-based prognostic predictors, statistical issues are more complex and there are more potential causes of poor reproducibility. A recent meta-analysis of 30 published studies of this type led to some interesting observations [33] . First, and not surprisingly, the authors found that significant associations were 3.5 times more likely per doubling of sample size and approximately 10 times more likely per 10-fold increase in the number of genes represented on the microarrays used. However, when they examined the reported performance of prognostic predictors as assessed by cross-validation (a method which tests a given prognostic predictor on portions of the original dataset that was used to develop it), they noted a paradoxical trend for smaller studies to show better sensitivity and specificity of their predictors compared to larger studies. This clearly highlights the statistical artifacts introduced into predictive expression profiling studies by relatively small patient numbers. They and others [34] note that cross-validation approaches are often incomplete, i.e. incorrectly performed, which can lead to inflated estimates of the sensitivity and specificity of a given microarray-based prognostic predictor. Other data analysis and interpretation pitfalls are discussed in detail by Simon et al. [34] .

The different results obtained by different groups may be due to technical factors but the variability may also have a biological component in that the differences may also reflect the likely "multidimensional" nature of prognostically significant genes or pathways, i.e. the presence of a matrix of several independent and additive prognostic categories for given cancer types. Larger more systematic and more standardized studies will be needed to sort out these variability issues. Nonetheless, for robust high quality gene expression signatures, such as those defined in acute leukemias [13] , reproducibility appears excellent, as demonstrated in a recent study that found excellent accuracy for microarray-based classification of acute myeloid leukemias at a central referral laboratory, regardless of differences in sample shipment time, storage time, or preparation period [35] . Finally, a major recently completed inter-laboratory comparability study of expression profiling using Affymetrix chips is also cause for optimism [36] . Four laboratories studied the same set of samples on Affymetrix U133A chips and found excellent interlaboratory correlations, only marginally weaker than the intralaboratory correlations. As expected, correlations were the weakest for genes with low expression levels and low variability across samples. They also noted that interlaboratory variability in microarray results was due less to handling or RNA extraction than to probe preparation and array hybridization and processing. Overall, their findings are encouraging and suggest that reproducibility issues will be effectively overcome by standardization of methods, careful QA/QC, and the use of a common microarray platform, allowing the clinical implementation of these assays on frozen surgical pathology material in the near future.

Acknowledgement
The author thanks Dr. William Gerald for sharing pre-publication data.

References

1. Schena M, Shalon D, Davis RW, Brown PO: Quantitative monitoring of gene expression patterns with a complementary DNA microarray. Science 1995, 270:467-470.
   
   
2. Lockhart DJ, Dong H, Byrne MC, Follettie MT, Gallo MV, Chee MS, Mittmann M, Wang C, Kobayashi M, Horton H, Brown EL: Expression monitoring by hybridization to high-density oligonucleotide arrays. Nat Biotechnol 1996, 14:1675-1680.
   
   
3. Pollack JR, van de Rijn M, Botstein D: Challenges in developing a molecular characterization of cancer. Semin Oncol 2002, 29:280-285.
   
   
4. Ladanyi M, Gerald WL (editors). Expression profiling of human tumors. Diagnostic and research applications. Humana Press, Totowa, NJ, 2003, 400 pages.
   
   
5. Ebert BL, Golub TR: Genomic approaches to hematologic malignancies. Blood 2004, 104:923-932.
   
   
6. van de Rijn M, Gilks CB: Applications of microarrays to histopathology. Histopathology 2004, 44:97-108.
   
   
7. Golub TR, Slonim DK, Tamayo P, Huard C, Gaasenbeek M, Mesirov JP, Coller H, Loh ML, Downing JR, Caligiuri MA, Bloomfield CD, Lander ES: Molecular classification of cancer: class discovery and class prediction by gene expression monitoring. Science 1999, 286:531-537.
   
   
8. Khan J, Wei J, Ringnér M, Saal LH, Ladanyi M, Westermann F, Berthold F, Schwab M, Antonescu CR, Peterson C, Meltzer PS: Classification and diagnostic prediction of cancers using gene expression profiling and artificial neural networks. Nat Med 2001, 7:673-679.
   
   
9. Su AI, Welsh JB, Sapinoso LM, Kern SG, Dimitrov P, Lapp H, Schultz PG, Powell SM, Moskaluk CA, Frierson HF, Jr., Hampton GM: Molecular classification of human carcinomas by use of gene expression signatures. Cancer Res 2001, 61:7388-7393.
   
   
10. Ramaswamy S, Tamayo P, Rifkin R, Mukherjee S, Yeang CH, Angelo M, Ladd C, Reich M, Latulippe E, Mesirov JP, Poggio T, Gerald W, Loda M, Lander ES, Golub TR: Multiclass cancer diagnosis using tumor gene expression signatures. Proc Natl Acad Sci U S A 2001, 98:15149-15154.
    
    
11. Bittner M, Meltzer P, Chen Y, Jiang Y, Seftor E, Hendrix M, Radmacher M, Simon R, Yakhini Z, Ben-Dor A, Sampas N, Dougherty E, Wang E, Marincola F, Gooden C, Lueders J, Glatfelter A, Pollock P, Carpten J, Gillanders E, Leja D, Dietrich K, Beaudry C, Berens M, Alberts D, Sondak V, Hayward N, Trent J: Molecular classification of cutaneous malignant melanoma by gene expression profiling. Nature 2000, 406:536-540.
    
    
12. Sorlie T, Perou CM, Tibshirani R, Aas T, Geisler S, Johnsen H, Hastie T, Eisen MB, van de RM, Jeffrey SS, Thorsen T, Quist H, Matese JC, Brown PO, Botstein D, Eystein LP, Borresen-Dale AL: Gene expression patterns of breast carcinomas distinguish tumor subclasses with clinical implications. Proc Natl Acad Sci U S A 2001, 98:10869-10874.
    
    
13. Yeoh EJ, Ross ME, Shurtleff SA, Williams WK, Patel D, Mahfouz R, Behm FG, Raimondi SC, Relling MV, Patel A, Cheng C, Campana D, Wilkins D, Zhou X, Li J, Liu H, Pui CH, Evans WE, Naeve C, Wong L, Downing JR: Classification, subtype discovery, and prediction of outcome in pediatric acute lymphoblastic leukemia by gene expression profiling. Cancer Cell 2002, 1:133-143.
    
    
14. Perou CM, Sorlie T, Eisen MB, van de Rijn M, Jeffrey SS, Rees CA, Pollack JR, Ross DT, Johnsen H, Akslen LA, Fluge O, Pergamenschikov A, Williams C, Zhu SX, Lonning PE, Borresen-Dale AL, Brown PO, Botstein D: Molecular portraits of human breast tumours. Nature 2000, 406:747-752.
    
    
15. Dairkee SH, Puett L, Hackett AJ: Expression of basal and luminal epithelium-specific keratins in normal, benign, and malignant breast tissue. J Natl Cancer Inst 1988, 80:691-695.
    
    
16. Nielsen TO, Hsu FD, Jensen K, Cheang M, Karaca G, Hu Z, Hernandez-Boussard T, Livasy C, Cowan D, Dressler L, Akslen LA, Ragaz J, Gown AM, Gilks CB, van de RM, Perou CM: Immunohistochemical and clinical characterization of the basal-like subtype of invasive breast carcinoma. Clin Cancer Res 2004, 10:5367-5374.
    
    
17. Yunis JJ, Mayer MG, Arnesen MA, Aeppli DP, Oken MM, Frizzera G: Bcl-2 and other genomic alterations in the prognosis of large cell lymphoma. New Engl J Med 1989, 320:1047-1054.
    
    
18. Offit K, Koduru PRK, Hollis R, Filippa DA, Jhanwar SC, Clarkson BD, Chaganti RSK: 18q21 rearrangement in diffuse large cell lymphoma: incidence and clinical significance. Br J Haematol 1989, 72:178-186.
    
    
19. Gascoyne RD, Adomat SA, Krajewski S, Krajewska M, Horsman DE, Tolcher AW, O'Reilly SE, Hoskins P, Coldman AJ, Reed JC, Connors JM: Prognostic significance of Bcl-2 protein expression and Bcl-2 gene rearrangement in diffuse aggressive non-Hodgkin's lymphoma. Blood 1997, 90:244-251.
    
    
20. Huang JZ, Sanger WG, Greiner TC, Staudt LM, Weisenburger DD, Pickering DL, Lynch JC, Armitage JO, Warnke RA, Alizadeh AA, Lossos IS, Levy R, Chan WC: The t(14;18) defines a unique subset of diffuse large B-cell lymphoma with a germinal center B-cell gene expression profile. Blood 2002, 99:2285-2290.
    
    
21. Friend SH: How DNA microarrays and expression profiling will affect clinical practice. BMJ 1999, 319:1306-1307.
    
    
22. Dumur CI, Nasim S, Best AM, Archer KJ, Ladd AC, Mas VR, Wilkinson DS, Garrett CT, Ferreira-Gonzalez A: Evaluation of quality-control criteria for microarray gene expression analysis. Clin Chem 2004, 50:1994-2002.
    
    
23. Shi L, Tong W, Goodsaid F, Frueh FW, Fang H, Han T, Fuscoe JC, Casciano DA: QA/QC: challenges and pitfalls facing the microarray community and regulatory agencies. Expert Rev Mol Diagn 2004, 4:761-777.
    
    
24. Sotiriou C, Khanna C, Jazaeri AA, Petersen D, Liu ET: Core biopsies can be used to distinguish differences in expression profiling by cDNA microarrays. J Mol Diagn 2002, 4:30-36.
    
    
25. Ojaniemi H, Evengard B, Lee DR, Unger ER, Vernon SD: Impact of RNA extraction from limited samples on microarray results. BioTechniques 2003, 35:968-973.
    
    
26. Knudtson KL, Griffin C, Brooks A, Iacobas A, Johnson K, Khitrov G, Lilley K, Massimi A, Viale A, Zhang W, Bao Y, Grills GS, Thaler H, Cox C. Factors contributing to variability in DNA microarray results: the ABRF Microarray Research Group 2002 study. www.abrf.org/ResearchGroups/Microarray/Publications/MARG_2002_Poster.pdf
    
    
27. Lee ML, Kuo FC, Whitmore GA, Sklar J: Importance of replication in microarray gene expression studies: statistical methods and evidence from repetitive cDNA hybridizations. Proc Natl Acad Sci U S A 2000, 97:9834-9839.
    
    
28. Iacobuzio-Donahue CA, Ashfaq R, Maitra A, Adsay NV, Shen-Ong GL, Berg K, Hollingsworth MA, Cameron JL, Yeo CJ, Kern SE, Goggins M, Hruban RH: Highly expressed genes in pancreatic ductal adenocarcinomas: a comprehensive characterization and comparison of the transcription profiles obtained from three major technologies. Cancer Res 2003, 63:8614-8622.
    
    
29. Woo Y, Affourtit J, Daigle S, Viale A, Johnson K, Naggert J, Churchill G: A comparison of cDNA, oligonucleotide, and Affymetrix GeneChip gene expression microarray platforms. J Biomol Tech 2004, 15:276-284.
    
    
30. Tan PK, Downey TJ, Spitznagel EL, Jr., Xu P, Fu D, Dimitrov DS, Lempicki RA, Raaka BM, Cam MC: Evaluation of gene expression measurements from commercial microarray platforms. Nucleic Acids Res 2003, 31:5676-5684.
    
    
31. Mecham BH, Wetmore DZ, Szallasi Z, Sadovsky Y, Kohane I, Mariani TJ: Increased measurement accuracy for sequence-verified microarray probes. Physiol Genomics 2004, 18:308-315.
    
    
32. Lee JK, Bussey KJ, Gwadry FG, Reinhold W, Riddick G, Pelletier SL, Nishizuka S, Szakacs G, Annereau JP, Shankavaram U, Lababidi S, Smith LH, Gottesman MM, Weinstein JN: Comparing cDNA and oligonucleotide array data: concordance of gene expression across platforms for the NCI-60 cancer cells. Genome Biol 2003, 4:R82.
    
    
33. Ntzani EE, Ioannidis JP: Predictive ability of DNA microarrays for cancer outcomes and correlates: an empirical assessment. Lancet 2003, 362:1439-1444.
    
    
34. Simon R, Radmacher MD, Dobbin K, McShane LM: Pitfalls in the use of DNA microarray data for diagnostic and prognostic classification. J Natl Cancer Inst 2003, 95:14-18.
    
    
35. Kohlmann A, Schoch C, Dugas M, Rauhut S, Weninger F, Schnittger S, Kern W, Haferlach T: Pattern robustness of diagnostic gene expression signatures in leukemia. Genes Chromosomes Cancer 2004, 42:299-307.
    
    
36. Beer DG, Meyerson M, et al. Inter-laboratory comparability study of cancer gene expression analysis using oligonucleotide microarrays. Clin Cancer Res 2005 (in press)