Assessment of the quality, reliability, and readability of the websites about orthognathic surgery

INTRODUCTION

The internet has become a primary source of information for individuals seeking data on a range of health-related topics, including dental and medical treatments. The increasing trend is driven by the convenience and accessibility of online resources. A growing body of evidence indicates that individuals are more inclined to explore their medical conditions online than to engage in discourse with healthcare professionals. This is attributed to the ease of access and perceived anonymity of these digital platforms.^[1] However, the pervasiveness of online information carries the risk of misinformation, which can potentially result in suboptimal patient decisions, heightened anxiety, and even a delay in seeking appropriate treatment options.^[2] While websites can facilitate communication between patients and healthcare professionals, such evaluations not only ensure that patients receive accurate and reliable information but also empower clinicians to guide their patients toward trustworthy sources.^[3]

Furthermore, websites facilitate communication between patients and field specialists, thereby enhancing their understanding of complex procedures. However, research has shown that a significant proportion of available online content may be biased or inaccurate, which can adversely affect patient perceptions and lead to misinformed treatment decisions. To address this issue, several tools have been developed to evaluate the quality of online health information.^[4] Among these tools, the Quality Criteria for Consumer Health Information (DISCERN) instrument assesses the reliability and quality of treatment information, the Ensuring Quality Information for Patients (EQIP) toolkit evaluates the competency and transparency of websites, and the Journal of American Medical Association (JAMA) Benchmark examines the transparency of website authorship and disclosure.^[5-7] In addition, the Flesch Reading Ease Score (FRES) and Flesch-Kincaid Grade Level (FKGL) are used to assess the readability of website content, with higher FRES scores indicating easier readability and lower FKGL scores representing content suitable for lower grade levels.^[8,9]

Congenital and skeletal disorders in the jaw and face due to genetic, environmental, or iatrogenic factors impact individuals both functionally and esthetically. These disorders should be treated with fixed or removable orthodontic appliances. Still, in severe skeletal Class II and Class III cases, craniofacial anomalies, and the presence of asymmetry, orthognathic surgical planning may be required to achieve ideal facial esthetics. Orthognathic surgery impacts patients’ quality of life both pre- and post-operation and is extensive, often spanning several years.^[10] The decision to undergo this treatment is pivotal, especially considering it involves general anesthesia and carries a notable risk of complications.^[11]

Individuals often base their treatment decisions on information obtained from platforms such as social media, websites, and YouTube^TM. However, a literature review has found that the reliability and readability of content on YouTube and other websites have generally poor-quality scores.^[12-14] The availability of reliable and high-quality information from various online platforms can significantly facilitate clinicians’ tasks. Literature reviews reveal that researchers utilize various criteria for website evaluation. Aldairy et al. evaluated websites about orthognathic surgery using DISCERN analysis and reported that the information obtained from most websites had a low DISCERN score.^[15]

Engelmann et al. assessed EQIP analysis and reported a broad range of website scores, with some sites leading patients to form unrealistic expectations.^[16] Bavbek and Tuncer found that patients generally obtain low-quality information from the internet when researching medical websites.^[17] In light of this, the present study aims to fill this gap by evaluating the quality, reliability, and readability of a diverse set of websites that provide information on orthognathic surgery. By applying a combination of the DISCERN, EQIP, and JAMA tools, this study provides a more holistic evaluation of the current state of online information on this topic and highlights areas that need improvement to better serve potential patients and clinicians alike.

MATERIAL AND METHODS

Ethical consideration and assessment tools

Ethical Committee approval was unnecessary for the study as it did not involve human material or data. Data sourced from Google Trends (Google Trends, 2020, Alphabet, USA, https://trends.google.com) indicate a worldwide preference for “orthognathic surgery” and “jaw surgery” as top search terms [Figure 1]. The internet search was conducted in July 2023 using the search terms found with Google Trends “orthognathic surgery” and “jaw surgery” with VPN turned off, utilizing “Google^TM (Google, 2023, Alphabet, USA, https://www.google.com),” “Yahoo^® (Yahoo Inc., 1995, USA, https://www.yahoo.com),” and “Bing^TM (Microsoft, 2009, USA, https://www.bing.com)” search engines. In 2023, Google was the most preferred search engine, with a 92.62% preference rate. It is followed by Bing search engine (2.9%) and Yahoo search engine (1.15%), respectively.^[18]

Close

Figure 1:

The appearance of the search terms, orthognathic surgery, and jaw surgery, in Google Trends.

Export to PPT

An evaluation was conducted on the top 50 websites for every search engine and keyword.^[19] In the present study, the first 50 results that people are likely to view have been evaluated.^[15,16]

English websites related to the topic were preferred as inclusion criteria for the study. The study excluded repeated websites on different search engines, advertisement websites, YouTube videos, research articles, and informed consent documents. Websites were evaluated using DISCERN, EQIP, and JAMA assessment tools and were categorized according to the author or website owner. Furthermore, the readability of the websites was assessed using the FRES and FKGL tools.

RESULTS

The measurements were conducted by an orthodontist with at least 5 years of experience, and for intra-researcher consistency, 10% of the websites were measured again after 2 weeks. The intraobserver reliability, calculated using the intraclass correlation coefficient with a 95% confidence interval, indicated a high level of agreement, demonstrating the consistency and reproducibility of the measurements.

Three hundred websites were initially recorded for the study, and only 93 met the inclusion criteria [Figure 2]. The websites that met the criteria were categorized based on the field of the website owner or author. Fifty-four of the websites were managed by oral and maxillofacial surgeons, 11 by orthodontists, eight by plastic surgeons, seven by multidisciplinary clinics, and 13 by unidentified individuals or non-health professionals. A comparison of groups based on each question of the DISCERN instrument is shown in [Table 1]. Significant differences among groups were found in the answers to the third (P = 0.037), fifth (P < 0.001), sixth (P < 0.001), and 16^th (P = 0.043) questions. The third question had the lowest values on websites belonging to maxillofacial surgeons (4.02 ± 0.57), statistically significantly lower than those of orthodontists (4.45 ± 0.52). In the evaluation of the fifth and sixth questions, the Layperson group demonstrated significantly higher scores compared to all other groups. In addition, in the assessment of the 16^th question, the Layperson group’s scores were significantly higher than those of the maxillofacial surgery group, as detailed in [Table 1].

Close

Figure 2:

Flowchart for this study.

Export to PPT

The comparison of the DISCERN instrument among all website groups is presented in [Table 2].

Significant differences were found in DISCERN Section 1, DISCERN Section 3, DISCERN Total Score, and JAMA values. The highest values for answers to DISCERN Section 1 questions were found in the Layperson group. The lowest values for the answer to the DISCERN Section 3 question were in the maxillofacial surgeon group. The DISCERN total score values were lowest in the maxillofacial surgeon group and highest in the Layperson group. The highest JAMA scores were also found in the Layperson group. No significant differences were found in other quality criteria among groups.

The examination of the relationship between scores obtained from websites for quality criteria is shown in [Table 3]. DISCERN, EQIP, JAMA, and FRES values showed a positive correlation with each other, while FKGL values showed a negative correlation with all others.

DISCUSSION

This study aims to assess the reliability, quality, and readability of online content related to orthognathic surgery. Websites sourced through three distinct search engines were analyzed and classified according to the expertise area of the author or site owner. Utilizing the first section of the DISCERN Instrument to evaluate reliability, it was found that the reliability across all groups was moderate. Readability was generally categorized at very difficult levels, while the quality of the websites, as determined by EQIP, was rated as poor.

Based on the findings from the DISCERN analysis, it has been observed that the websites “refer to areas of uncertainty” and provide information about the conditions “what would happen if no treatment” is used at a lower level. In addition, it was noted that the websites provide information in a neutral manner and generally include information about individuals’ quality of life after surgery. According to our study’s results, websites owned or authored by maxillofacial and plastic surgeons were of poor quality. In contrast, those related to orthodontics, multidisciplinary clinics, or those created by unspecified authors or those not in the health field were of medium quality. Furthermore, on evaluating the overall scoring of the websites, they were found to have poor content. Aldairy et al. evaluated websites providing information about orthognathic surgery; the highest DISCERN score was 64, and the lowest was 21.^[15] In a study by Bavbek and Tuncer evaluating websites about orthognathic surgery in Türkiye, according to DISCERN analysis results, websites belonging to maxillofacial surgeons had an average score of 43.67, orthodontists’ websites averaged 28.72, and plastic surgeons’ websites scored 33.34.^[17] In our study, the highest score was 72, and the lowest was 23. Websites are designed to provide brief information about orthognathic surgery and direct patients to schedule a consultation with a specialist for detailed information. Hence, the outcomes of the research suggest that the websites’ quality ranges from moderate to poor.

In the EQIP analysis, the content quality of the websites was generally rated between 50% and 75%. Notably, websites owned by maxillofacial and plastic surgeons scored lower in the EQIP evaluation. The inclusion of alternative treatment options was found to be limited; while information on orthognathic surgery was prevalent, content on camouflage treatments or alternative surgical options was scarce. In addition, few websites referenced content from other sources. Although some websites featured patient experiences with orthognathic surgery, the majority lacked patient testimonials and data. Engelmann et al. reported an average EQIP score of 68.256 in their evaluation of orthognathic surgery websites.^[16] The reason for the value obtained in our study being lower may be the inclusion of more websites in our study.

In the JAMA analysis, it was reported that the general scoring of the websites was average, providing a general idea. This situation generally indicates that the citation and explanation rate on the websites is low. Furthermore, generally, the creation date of the websites is indicated. Individuals can get an idea about the content’s recency thanks to this data. The restricted number of websites disclosing their funding sources in the JAMA benchmark could be attributed to the nature of their creation. Websites on orthognathic surgery are typically informational platforms established by doctors, with minimal financial backing from commercial brands selling medical products. In addition, the general absence of references on these websites might be due to doctors relying on their expertise when disseminating information or perhaps overlooking potential plagiarism issues.

The readability evaluations using FRES and FKGL analyses reveal that the websites’ content is geared toward a college-level audience. The highest FRES score and easy readability were observed in the layperson group. When evaluating the FKGL results, it was seen that the websites have high-school readability levels, and the lowest FKGL score was seen in the Layperson group. When all websites are evaluated, the website contents are generally created suitable for high school and college levels. Pithon and Dos Santos, in their research evaluating websites on post-orthognathic surgery pain, reported that the readability level of the content was suitable for university graduates or undergraduate students.^[24] The findings are similar to the findings obtained in our study.

Considering the data obtained in the study, it was detected that some websites do not share their information sources and do not include current treatment options on their websites. In this context, including this data can increase the reliability and quality of the websites. Patients can be allowed to reach different perspectives by also including content found on different websites. At the same time, websites can enlighten uncertain topics and the risks of treatment. Given the varied educational backgrounds of individuals seeking information on orthognathic surgery, adopting simpler language can make the content accessible to a wider audience.

CONCLUSION

The findings of this study indicate that the quality of English-language websites on orthognathic surgery is generally low to moderate, and the readability is difficult for the average reader. Patients seeking information should not rely solely on these websites, as incomplete or misleading data may lead to misunderstandings. Therefore, patients need to consult healthcare professionals for accurate and comprehensive guidance. Clinicians, on the other hand, should proactively guide their patients by providing recommendations for reliable online resources and helping them critically evaluate the content they find online. This approach will not only mitigate the risk of misinformation but also empower patients to make more informed treatment decisions.