Patients with OTSCC treated surgically at Hutt Hospital between April 1997 and September 2012 were included in this study which was approved by the Central Regional Health and Disability Ethics Committee (Ref. no,: 12/CEN/74).

50 patients were identified from our prospectively maintained head and neck database. Demographic data of the patients and details of their tumors were obtained from the database. Patients were excluded if they had previously undergone radiotherapy (n=2), had a recurrent tumor following previous treatment (n=1), the tumor sample was unavailable (n=6) or the slides were inadequate (n=9). 32 OTSCC samples were available for the final analysis.

Tumor TNM stage, clinical stage, histological differentiation, and presence of perineural and/or lymphovascular invasion of each OTSCC were documented.

Hematoxylin and eosin (H&E) staining was performed on 4μm-thick formalin-fixed paraffin-embedded sections of all 32 OTSCC samples to confirm the presence of the tumor in the sections and appropriate histological grading by an anatomical pathologist (HDB). 3,3-Diaminobenzidine (DAB) immunohistochemical (IHC) staining of the sections was then performed using the Leica Bond auto-stainer (Leica, Nussloch, Germany) as previously described 18. Staining for CD34 (ready-to-use, cat# PA0212, Leica), ACE (1:40; cat# MCA2054AbD, Serotec, Kidlington, UK), ATIIR1 (1:25; cat# Ab9391, Abcam, Cambridge, MA, USA), ATIIR2 (1:2000; cat# NBP1-77368, Novus Biologicus, Littleton, CO, USA), diluted with BondTM primary antibody diluent (cat# AR9352, Leica) was performed on all tissue samples.

Positive human control tissues used to confirm the specificity of the primary antibodies were liver for ACE and ATIIR1, and kidney for ATIIR2 (data not shown).

The tumor was marked on the slide by an anatomical pathologist (HDB) and tumor angiogenesis was quantified by two independent observers (P1 and P2) by counting the CD34+ endothelial cells on the microvessels within and immediately adjacent to the tumor. The extent of tumor vasculogenesis was similarly quantified by counting cells lining the microvessels that stained positively for either ACE, ATIIR1 or ATIIR2. Each slide was read by two observers at 400x magnification to subjectively select three hotspot areas within each tumor that showed the greatest number of distinct positively stained microvessels. Each hotspot was then assessed using a 25-point Chalkley graticule (Olympus, Tokyo, Japan), in a 200x field on a light microscope (10x eyepiece, 20x objective, area 0.196 mm²) (Olympus). The graticule was then orientated so that as many points as possible were on or within the positively stained microvessels. The average counts of both observers three hotspots gave the total vascularity score used in the analyses.

To determine statistical significance a Pearson s correlation was performed using IBM SPSS v22.

H&E staining (Figure 1A) confirmed the presence of SCC on the slides. To determine if Chalkley counting was a reliable method of quantifying tumor angiogenesis and tumor vasculogenesis, Chalkley counting on CD34 (Figure 1B, brown), ACE (Figure 1C, brown), ATIIR1 (Figure 1D, brown) and ATIIR2 (Figure 1E, brown) by two independent observers (P1 and P2) was performed. The averages of their three ‘hot spot’ counts were compared with known prognostic factors chosen for this study: tumor TNM stage, and clinical stage (Suppl. Table 1). Due to the relatively small sample numbers we were unable to perform any meaningful correlations between histological differentiation, the presence of perineural and/or lymphovascular invasion, and Chalkley counting of OTSCC.

Chalkley counting as a method of quantifying tumor angiogenesis and tumor vasculogenesis to prognosticate cancer has been widely used across different cancer types, including breast 91319 and gastrointestinal 11 cancers. Hansen et al. 13 investigated different methods of quantifying tumor angiogenesis in breast cancer and reported that Chalkley counting produces the least observer variability. They used this method to study 836 breast cancer patients and concluded that Chalkley count is a reliable and independent prognostic tool for breast cancer 9. Despite this earlier study indicating Chalkley count having the least observer variability in selecting the microvascular hotspot (the most observer-dependent step), the College of American Pathologists regards quantifying microvessel density by Chalkley counting as being an unreliable prognostic tool for breast cancer 14.

Vascular mimicry leads to greater perfusion in cancer leading to tumor growth and metastasis 17. We had therefore chosen ACE in this study on OTSCC to quantify tumor vasculogenesis. ATIIR1 and ATIIR2 were also selected given their involvement in putative stem cell differentiation 16.

The observer-dependent selection step in Chalkley counting is highlighted in this study with only a moderate correlation observed for CD34, and no correlations between the two independent observers for ACE, ATIIR1 and ATIIR2. Furthermore, there was no significant correlation between each of the four markers and the prognostic factors of OTSCC chosen in this study. These results are consistent with the findings of Hannen and Riediger 20, showing the unreliability of similar methods for quantifying angiogenesis/vasculogenesis in predicting OCSCC. This may in part be due to the relatively small sample size used in this study and it remains the topic for further investigation.

Possible reasons for Chalkley counting being reliable for some (e.g., breast, gastrointestinal and prostate) cancers, but not for OTSCC, include tumor angiogenesis not being a suitable measure for determining prognosis in oral tissues which are typically vessel-rich 20. In addition, Chalkley counting shows great variation between observers and lacks reproducibility as a result of other factors such as differences in the size and number the observer-dependent hot spots during the selection step 1920. Furthermore, the appropriateness of using ACE, ATIIR1 and ATIIR2 as markers for tumor vasculogenesis, at least in OTSCC, remains to be conclusively determined. These factors contribute to a decrease in the overall reliability and validity of this method and bring in to question its role in the prognostic setting.

We applied Chalkley counting to quantify tumor angiogenesis using CD34 and tumor vasculogenesis using ACE, ATIIR1 and ATIIR2 but found this to be unreliable in OTSCC. Its weakness relates to the reproducibility (also known as reliability) of its results. Reliability is a criterion that is applied universally to measuring instruments and is generally obtained by correlating the results of repeated measurements on the same things by both the same and different people (co-efficients of equivalence) and/or at similar and different times (coefficients of stability). When both times and observers are different it yields the coefficient of stability and equivalence. As well as being important for obvious reasons, reliability is critical for its relationship to validity (the degree to which the measurement actually reflects the characteristics of what it is measuring). In almost all cases reliability indicates the maximum possible level of validity that can be obtained. Acceptable levels of reliability begin to be reached when its correlations are 0.7 or more - meaning more than about 50% of the variance is accounted for. Depending on the task, however, levels of reliability may need correlations in the range of 0.9 (meaning 80%+ of the variance is required to be accounted for). Averaging observations from a number of observers or a number of repeats will increase the reliability estimates to levels which can be predicted from the application of the Spearman-Brown Formula. In our observations we recorded correlations no greater than 0.5. These results indicate that Chalkley counting is not suitable for our measurements procedures.

The lack of statistical significance between the independent observers, and the known prognostic factors we have chosen with the markers presented in this study, leads us to conclude that Chalkley counting is not a suitable method for quantifying tumor angiogenesis and vasculogenesis in OTSCC, and therefore, an unreliable prognostic tool.

The limitations of this study include the relatively small cohort. Future work requires a larger sample size and the use of other marker of angiogenesis, such VEGFR2, however, this remains the topic of further investigation.