Does the content quality of YouTube videos about aligners differ from the perspectives of dentists and orthodontists?

INTRODUCTION

Transparent aligners were introduced for use in orthodontic treatment, following Dr. Harold Kesling’s first use of a tooth positioner (Essix plate) in 1946 to achieve tooth movement.^[1] Clear aligners have become increasingly popular since the advent of dental materials and 3D technologies in the past 15 years.^[2,3] In particular, clear aligner technology has been widely used in orthodontics with the start of Invisalign (Align Technology, San Jose, Calif) production after 1999.^[4] Analyzing a Google search done in late 2015, it was determined that there are about 27 different clear aligner treatment brands.^[5] Aligners in orthodontic treatments provide better outcomes in terms of patient discomforts in daily functions, esthetic, and psychosocial concerns, eventually quality of life.^[6] The major disadvantages are that it does not applicable in the treatment of all kinds of malocclusion and its high cost.^[7] Nowadays, web-based platforms have become increasingly accepted and widely used tools for obtaining health-related information by the public as well as face-to-face interviews between patients and health-care professionals.^[8,9] There is a relationship between health literacy level and web usage rate.^[10] YouTube, which has become the second most popular website in the world since its establishment in 2005, is a web-based video sharing platform that also includes videos containing health information. Since Y Kindly check the Edits made. ouTube includes both visual and auditory materials, it is frequently preferred by patients to seek information about treatment outcomes in cases where esthetic expectations such as dental treatments.^[11-13] However, most studies reported that YouTube^™ videos contain scientifically inaccurate, misleading, and not up-to-date health-care information that adversely affects the patient.^[11,14,15] The main reason for this situation, which is expressed by health professionals as a concern regarding web-based platforms, is that videos shared on YouTube are not subjected to any peer review process based on the principle of freedom of expression.^[16] In the light of all this information, the aim of this study was to evaluate the content of YouTube videos about clear aligner treatments by orthodontists and dentists. The auxiliary aim was to evaluate the relationship between obtained data such as temporal distribution, video length, uploader of video (dental profession, commercial, and individual), and viewer attitudes (number of views, number of likes, number of dislikes, and number of comments).

MATERIAL AND METHODS

This cross-sectional YouTube video content analysis study had no need of ethical committee approval since public access to videos on YouTube has no restriction.

Before the YouTube video analysis, Google Trends (https://trends.google.com) data were used to find out which of the aligner terms were searched most frequently within a certain period. The Google Trends website (https://trends.google.com) was used to determine, in which aligner term was searched most frequently over a specific period. To obtain accurate data searching period, parameter was restricted to the “Worldwide” with the past 5 year filter among the most popular keywords, the terms “aligner; aligners; clear aligner; plastic trays; transparent plates” were chosen to be searched on Google Trends [Figure 1] (August 23, 2021). Considering data provided by Google Trends, the “aligner” is the most common term searched on the web. To hinder any influence on outcomes by previous activities of users on YouTube, a new YouTube user account was registered. Videos related to “aligner” were searched on the YouTube website (https://www.youtube.com) on August 23, 2021, listed with “sort by relevance” option, which was determined by YouTube using a combination of factors such as number of views, ratings, and upload dates. Considering the fact that 92% of users in a report on users’ behavior in web-based search engines take into account search results listed on the first three pages,^[17] a total of 80 videos as listed in the first three pages were included to study. Following, a playlist called as “aligner” was organized on YouTube and Source locators (Uniform Resource Locators) of videos were saved. The 80 videos were viewed and evaluated by the clinician (E.B). These viewings took place between 09:00 am and 12:00 noon throughout the week with a break of at least 3 min between two consecutive videos.^[11] Daily viewed video counts was restricted up to 8/day. These timing details mentioned above were managed by the Supervisor (S.S). According to the exclusion criteria listed below, only 39 of the 80 videos remained for content analysis. The exclusion criteria were as follows:

1. The video language not being English

2. The video was lacking audio content and/or title information

3. Irrelevant content to the term of “aligner”

4. Videos produced for commercial purposes

5. Duplicated and/or sliced videos

6. Videos longer than 15 min^[11,18] [Figure 2].

Close

Figure 1:

Determining of keyword on Google Trends website.

Export to PPT

Close

Figure 2:

Inclusion and exclusion criteria.

Export to PPT

Following, a unique number between 1 and 39 was assigned to each video and a computer-generated randomization table was produced by S.S. Then, 39 videos with URL were saved into an external memory device with the randomized list. This external memory device was given to an independent researcher (S.E.M) to manage the rest of the study period. An independent orthodontist and an independent dentist working at Istanbul Aydin University Oral and Dental Health-care Application and Research Center were selected to evaluate the content of these 39 videos. The selection was performed randomly by the S.E.M from seven orthodontists and four dentists who met the following criteria. These criteria were as follows;

• To work as a dentist for at least 5 years

• To hold a validated certificate related to clear aligners for at least 1 year

• And not to have uploaded videos about clear aligners on any web-based media platform before

All videos were viewed by independent viewers during the study period under the following conditions described by Meseli et al.^[11]

• Watched between 09:00 am and 12:00 noon

• There was a break of at least 3 min between two consecutive videos

• At least 7 h of sleep per day

• A well-lit and regularly ventilated environment at normal room temperature

• No external stimuli such as cell phones, computers, or televisions

• A standard distance of 1 meter from the screen.

The following general video informations were recorded as parameters for each video: date of uploading, source of video, number of views, number of likes, number of dislikes, number of comments, duration of the video, interaction index (%), and viewing rate (%). Videos were classified into three distinct groups in consider with their uploaders: (a) dental professional, (b) dental company, or (c) other individuals such as YouTubers. The video content quality (VCQ) score,^[11,19] video interaction and quality index (VIQI) score,^[19,20] reliability score,^[21] and global quality scale (GQS) score^[22] were recorded to achieve an assessment of the video content objectively.

The VCQ score was calculated for each video with the following nine parameters about clear aligners: definition of appliance, duration of treatment, application procedures, treatment process, comparison of treatment methods, oral hygiene rules, side effects, treatment fees, and biomechanics. For each video, each of these nine parameters was scored based on consensus judgments on a scale of 0 to 3. A score of,

• 0: Misleading or no information in video

• 1: Insufficient information in video

• 2: Sufficient information in video

• 3: Extensive information in video on the topic.

Each of these parameters was defined as below to viewers before study;

• Definition of appliance: Introducing appliances as aligner that is used in orthodontic treatment and that patients can put on and take off on their own

• Duration of treatment: Information about skeletal developmental stages and mixed dentition

• Application procedure: The aligner should be used throughout the day, except for eating

• Treatment process: Explaining the aligner procedures from the early impressions gained before starting treatment to the end phase

• Comparison of treatment methods: Explaining the advantages of employing aligners, such as its being easier to use and clean, and less likely to cause tissue irritation compared to fixed appliances

• Oral hygiene rules: Explaining the effective cleaning methods of the aligners

• Side effects: defining the side effects such as how it affects speech and causes unwanted, uncontrolled tooth movement

• Treatment fees: Aligners are more expensive than other treatment methods/average wage

• Biomechanics: Effective use of the necessary attachments and equipment to provide the required tooth movement.

Following, the above said nine parameters for each video were recorded between 0 and 3 scores. The VCQ value of a video was calculated by sum of them; therefore, VCQ value could range between 0 and 30. A video with a total score of 30 points contains scientifically extensive information about the relevant topic.

VIQI was based on the flow of information, accuracy of information, video quality, and the sensitivity (level of agreement between title of video and the content). This five-point Likert scale designed parameter was performed to evaluate the overall quality of each video with the points are as follows;

• 1: Poor quality and poor flow

• 2: Generally poor quality and poor flow

• 3: Moderate quality and suboptimal flow

• 4: Good quality and generally good flow

• 5: Excellent quality and flow.

The videos included in the study were scored according to reliability scores. Higher score indicates the more reliable video content. A survey, which consists of 5 yes/no questions detailed below, was performed. Moreover, considering given answer, they were scored with 0 (no) or 1 (yes) point. The questions of the survey were as follows;

1. Are the goals clear and achieved?

2. Are reliable sources of information used? (Publication cited, the speaker being an orthodontist)

3. Is the information presented consistent and unbiased?

4. Are additional sources of information listed for patient reference?

5. Are areas of uncertainty mentioned?

Moreover, the GQS, a 5-point Likert scale, was used to determine the quality of the videos. The scale was designed so that the total score for each video was as follows: one dot for very low quality, two dots for low quality and limited use, three dots for average quality, four dots for good quality, and five dots for very good quality.

In the final stage, the number of views, the time between upload and viewing, likes and dislikes, and the duration of the videos were recorded to obtain the viewership rates of the videos and viewers’ interactions with the videos. The formulas introduced by Hassona et al.^[23] were used for this purpose [Figure 3].

Close

Figure 3:

The formulas used in the study for interaction index and viewing rate.

Export to PPT

RESULTS

In this study, 80 videos shared on YouTube for the keyword “aligner” were evaluated considering the exclusion criteria detailed above, only 39 videos from them were taken to under consideration for the study [Figure 2]. The descriptive statistical data of the included 39 videos revealed that average number of views was approximately 15.5 K, and the least viewed video was viewed by 116 times. In addition, median values of number of likes, number of dislikes, and number of comments were 77, 4, and 8, respectively [Table 1].

[Table 2] figured out that the comparison of the data obtained from videos in terms of by their uploader sources. Accordingly, the number of videos uploaded by dental companies is the highest among the included videos, followed by videos uploaded by dental professionals with 15 videos. In the comparison of the variables according to the uploader sources, it was found that the number of comments (average for number of comments = 31) and interaction index (median = 0.93) of the videos shared by dental professionals were significantly higher compared to the other two groups. In the evaluation of the videos, it was concluded that the VCQ, VIQI, and GQS averages recorded both by orthodontists and by dentists were similar according to the uploader source. On the other hand considering the average reliability scores recorded by orthodontists, reliability scores of the uploaded videos by other individuals were significantly lower than the both other uploader sources (for both P = 0.017). However, there was no significant difference in the average reliability scores of videos scored by dentists in terms of video uploaders sources (P > 0.05) [Table 2].

[Table 3] shows the agreement between the scores recorded by the orthodontist and dentist who evaluated the videos regarding the reliability, quality, and content of the videos. To reveal the interobservers agreement level in cases where one of the observers is not accepted as a reference (the gold standard), the level of agreement increases as the CCC value approaches “1,” which is the method defined by Barnhart et al.^[24] Based on the scores for the evaluated video variables given by the orthodontist and dentist, interobserver agreement was “good” for the VIQI and GQS variables, whereas it was “very poor” for the reliability score and VCQ.

The associations of the variables evaluated on the videos were displayed on [Table 4]. Accordingly, each of the variables of number of views, number of likes, number of dislikes, and number of comments showed a significant same-directional relationship with each of the remained of them (for each correlation P = 0.000). Similar to this, viewing rate had significant positive association with the each of the above-mentioned variables (for each correlation, P = 0.000). In addition, while the relationship between the interaction index and the number of likes variable was positive (P < 0.05), between the time since uploaded variable was negative (P < 0.05) [Table 4].

DISCUSSION

In the globalizing world, people seek health-care informations and approaches that could enhance the well-being on web-based applications and websites. As one of these websites YouTube^™ provides the uploaded videos included health-related ones by the own registered users to the public and to seek health-related informations, it is becoming increasingly popular among patients as well.^[25] Since the health-related information presented in these videos is not peer-reviewed, it does not always provide accurate and sufficient information to patients and may even provide incorrect, irrelevant, and harmful information.^[14] With the developing technology, these are important issues that should not be forgotten in the design of metaverse that offer open access to accurate information using large technobytes, where digital data are stored.^[26]

Videos on aligner treatments were viewed by both the orthodontist and the dentist to compare content quality of videos. The content quality of the videos was found similar among all source of uploaders. On the other hand, the outcome of that level of agreement between dentist and orthodontist in terms of VCQ shows that the content analysis of the video can be affected by the characteristics of the viewer. Considering the nature of the human-being perceptions and perspectives differ person to person, this is an expected outcome, even the viewer is an expert or not in the field. For example, a treatment plan with aligners causes a complication means as unsuccessful outcomes on treatment by orthodontist, whereas it means as just disregardable complication of treatment by another orthodontist or dentist. Therefore, information and knowledge of the viewer are as important as video uploader source.

Furthermore, VCQ outcomes showed that similar median values revealed from both viewers and the videos had moderate level information. The interpretation of similar VCQ values with a poor level of agreement indicates that the viewers scored information counts in the videos were same counts, but they were in different categories. The main reason for insufficient content quality of videos is that everyone can upload videos on YouTube without any peer-review process.^[11] Even though, lackness of well-described standards for uploaded videos on YouTube, audiovisual quality, and flow of the videos were found acceptable level for both viewer. The GQS findings of the videos about the aligners were consistent with the outcomes from the study on YouTube video content analysis.^[13,27]

In other studies, it has been stated that although the content quality of YouTube videos is insufficient, these videos have positive effects on people’s knowledge and awareness levels on health-related issues. Al-Silwadi et al. reported that social media tools that convey audio-visual information on YouTube^™ are beneficial in increasing the knowledge level of patients undergoing fixed orthodontic treatment.^[28] Another study that was conducted by Coban and Buyuk determined that videos about distraction osteogenesis should be analyzed since YouTube^™ is the most popular platform for audio-visual content on social media. As a result of abovementioned study, although the videos on craniofacial distraction osteogenesis have poor-quality content and reliability, it was stated that the videos alone provide more information than verbal narration.^[29]

According to the analysis of videos based on the source of upload, the most are uploaded by dental companies. The reliability scores of the orthodontist were lower for videos uploaded by the person with no relation with dentistry. Similar to this Bozkurt et al., the videos uploaded by health-care professionals are more reliable than other sources.^[18] It is evident from these outcomes that health-care providers or related person to this area could produce more reliable videos due to comprehensive content provided by them. Considering the existence of the vast video uploaders, to refer the their patients to the accurate sources, knowledges of the clinicians must be up to date about the sharings on social media.

Besides the strengths sides of it, this study has some limitations. YouTube is a platform that existed videos in it changes in every seconds. However, this study was planned as cross-sectional that evaluation was conducted on one point in timeline. Consequently, the outcomes could not reflect a causality and could not have a generalizability. In addition, the videos were evaluated by only one orthodontist and one dentist. It is possible to achieve more reliable results with the studies designed with the increased number of viewers. The present study included only English videos. Therefore, information in other languages was not evaluated. Nevertheless, there are many videos in other languages that could be useful with different content that focuses on a wide range of topics. Therefore, it is important to evaluate the information content of YouTube videos in different languages as well.

CONCLUSION

There are many videos on YouTube^™, one of the most popular social media platforms, which cover various topics related to aligner videos. In view of the fact that the videos were evaluated by both an orthodontist and a dentist on the topics considered to be beneficial for patients, there are not enough good-quality videos in terms of content. Thus, YouTube^™ videos do not provide adequate information regarding orthodontic treatment with aligners. Orthodontists should create more high-quality content and reliable videos on YouTube^™ to ensure that patients have access to accurate and reliable information about aligners more easily. Moreover, considering the significant extent of internet usage nowadays, health-care professionals should be more competent and more involved in social media platforms to increase the level of health literacy in public.