The IRCTP portal frequently updates its database. However, IRCTP's search portal was accessed in late December 2019. Therefore, the dataset will only cover clinical trials until 2019. A single keyword query was used with the word “Tuberculosis” to search the database. All result outputs were exported in the file format of comma-separated value (.csv) and renamed “core_data.csv.” As shown in Table 1, the forty fields were exported in the result. An example (regardless of any particular clinical trial) of each field is also shown in the table. This dataset covers a total of 1635 clinical trials of tuberculosis.

Necessary data refining was done manually, without any disruption to its scientific significance. For example, “male” and “males” both are manually corrected as “Male” in the field of gender. In the same way, “Intervention” and “Interventional” both are corrected as “Interventional” in the area of the study type. Microsoft Excel was utilized for this data preprocessing, and Tableau^® Desktop (Trial Version) was utilized for data analysis and visualization.

Each row of “core_data.csv” represents one individual clinical trial, as shown in Fig. (2). The text mining widget's “Import Document” tool in the Orange software can handle data only in text (.txt) file format. Therefore, an individual row (from core_data.csv) must be exported to the individual text file for the preparation of the corpus. This text file must be renamed with the respective clinical trial “Trial ID.” The first row of the dataset, for example, is exported as a text file renamed as “ACTRN126100643077.txt.”

For the text mining purpose, only “Public title” and “Scientific title” were exported into the text file, which embraces the scientific meaning of the clinical trial. Thus, each text file contains the titles of its respective clinical trial. A macro function of Microsoft Excel was used for the export of each row containing only public and scientific titles in a single text file and further renaming it with its “Trial ID.” This execution has generated a total of 1635 text files, each containing the public and scientific title of their respective clinical trials.

As shown in Fig. (3), all text files, according to their source registry, were grouped into separate subfolders. These subfolders have been renamed according to their respective source registries. These sub-folders were renamed as per their respective source registry. The source registry ultimately represents an individual country where the particular clinical trial was registered. Therefore, a total of 15 subfolders with text files of their respective clinical trials were created in this way. As shown in Fig. (3), each sub-folder named after the source register represents their respective countries. All these sub-folders were stored in one principal folder named “Corpus”. This principal folder was used in the Orange software to create a corpus for further text mining.

This “Corpus” folder was browsed and selected using the Orange software's “Import Documents” tool. In Orange, each of the fifteen subfolders is referred like 15 different categories. The text files are grouped in subfolders and look like a structured dataset because they are marked with their respective “source registry.” The textual dataset in those subfolders, however, is unstructured from the perspective of text mining. The entire dataset is available from the GitHub (https://github.com/Swayamprakashpatel/Tuberculosis_Clinical_Trial_TextMining.git).

Fig. (4) shows a basic workflow for text mining in the Orange software. The Import Documents tool is connected to the Corpus tool to generate a corpus. Corpus viewer, preprocess text, bag of words, distance, hierarchical clustering, distribution, and word cloud and data table widgets were connected in series as shown in the figure.

Before the analysis, various text preprocessing was performed using the “Text Preprocessing” tool in the Orange.

Table 2 shows all of the options available in the “Text Preprocessing” tool. All words were transformed into lowercase, and if present, their accents were removed. By enabling the “parse Html” option, the HTML markups were removed from its text. URLs of the text data were simply disregarded. For the tokenisation, only the word and punctuation were taken into account. Words were lemmatized using “WordNet” as a lexical database instead of mere stemming. N-gram was set to a maximum of two (N=2) words.

Besides basic stop words and regular expressions in English, some other words had also been filtered from the text. These filter words are 'tuberculosis;' 'tb;' 'exclusion;' 'inclusion;' 'criteria;' 'treatment;' 'study;' 'patient;' 'therapy;' 'medicine;' 'effect;' 'clinical;' 'trial.' These are the most common words in any clinical trial and do not constitute the scientific sense or meaning of any clinical study (at least in this work). We recommend reading the tutorials provided by the Orange software over its website for a better understanding of this tool.

For text mining, a technique of cluster analysis was used to group the clinical trials with the common research interest. The hierarchical clustering (HC) technique was utilized to understand the various research interests among the registered clinical trials of tuberculosis. Hierarchical Cluster Analysis (HCA) uses an algorithm that keeps similar objects in common groups called clusters [21-24]. Depending upon the most occurring words, they are grouped in a common cluster. The top-down strategy has been used in this work to ensure effective clustering [25]. “Distance” and “Hierarchical clustering” are the widgets in the orange software which were used for the cluster analysis. Distance between the rows and a cosine distance matrix was utilized in the hierarchical clustering. Ward's method was chosen to cluster the clinical trials [26] effectively, and the dendrogram height ratio was set at 35 percent of the total height for the selection of clusters.

Fig. (2). Core data in comma-separated (.csv) file format (only the first few columns and rows shown here).

Fig. (3). Grouping of text files of the clinical trial as per respective source register in appropriate folders.

Fig. (4). Exemplary workflow of text mining in orange.

As shown in Error! Reference source not found., the “hierarchical cluster” (HC) tool was connected to the “Distribution,” “Word Cloud,” “Data Table,” and “Contingency Table” tools in the series manner. “Selected Data-to-Data” and “Selected Data-to-Corpus” options were set as the connection between HC, distribution, and word cloud tools, respectively. At the same time, “corpus-to-data” and “data-to-data” connections were utilized for word cloud, data table, and contingency table tools. For the analysis of each cluster, the most common words were selected from the word cloud one by one until it covered all clinical trials in the contingency table. This list of words represents that particular cluster (Refer to Supplementary Table 1). The meaningful research area was concluded manually from this list of words and considered as a common research interest of the clinical trials within the respective cluster.

In some clusters, the resolution of words for firm mapping of research interest is poor. We can not conclude a common research interest from their list of most frequent words. In such cases, it is imperative to perform sub-clustering. The process of sub-clustering is simple and similar to hierarchical clustering. It includes the selection of clusters from the distribution tool for which sub-clustering is required. Connect the distribution widget further to the new hierarchical clustering widget, followed by a new distribution widget and a new word cloud widget in series. Mapping of research interest can be further performed for the subclusters in a way similar to that performed for its parent cluster.

As shown in Fig. (11), a total of 41 clusters were identified and selected for further analysis via the hierarchical clustering technique.

The supplementary table (Table 1) in the appendix represents the research interest of clinical trials of every cluster in a comprehensive manner. Out of 41 clusters, only seven clusters need sub-clustering to understand the research interest of their clinical trials. Thirty-three (33) clusters, for which successful mapping of the research area was performed, covering 1015 (Supplementary Table 2) clinical trials out of a total of 1635. The remaining 620 clinical trials, consisting of C27, C29, C32, C34, C35, C37, and C40, may be mapped by sub-clustering for their research interest. This sub-clustering and analysis are not shown here since it can extend the usual publication limit.

In the supplementary table (Table 1), the research interests or areas mapped for 1015 clinical trials are summarized and shown in Table 5. As per this summary, the highest number of TB clinical trials are focused on HIV infection. In addition to common research trends on HIV, vaccine, BCG, MDR, etc., other interesting research trends including vitamins, active case findings in the community, diabetes, pleural effusion, fixed-dose combinations, meningitis, etc. are also mapped.

Cluster 35 contains 384 clinical trials for which the mapping of research interest is not possible without sub-clustering. Sub-clustering of C35 (similar to hierarchical clustering) and the selection of 80% of the total height of the dendrogram have grouped these 384 clinical trials in 14 different Sub-clusters. As shown in Fig. (12), these 14 subclusters have a positive silhouette score. It means the mapping of research areas for these clusters is possible with excellent resolution. The research area of 8 clusters out of these 14 was identified and listed in Table 6. Rest can be easily mapped through re-reclustering (not shown here).

Fig. (9). Word cloud of text from tuberculosis clinical trials.

Fig. (10). Word cloud of drugs mentioned in tuberculosis clinical trials.

Fig. (11). Hierarchical cluster distribution.

Fig. (12). Silhouette plot of subclusters of C35.