Student, Kazakh-British Technical University, Kazakhstan, Almaty
MODELING AND DEVELOPMENT OF ONTOLOGY-BASED INFORMATION SYSTEM TO SUPPORT MEDICAL ACTIVITY
УДК 004.89:61
ABSTRACT
Healthcare systems increasingly require intelligent knowledge management solutions capable of supporting complex clinical decision-making. Traditional database-driven medical information systems lack the semantic expressiveness needed to capture the nuanced relationships inherent in clinical knowledge. Objective: This paper presents the modeling and development of an ontology-based information system (OBIS) designed to support medical activity through formal knowledge representation, semantic reasoning, and clinical decision support. Methods: A five-layer ontology architecture was designed using OWL-DL and SWRL, encompassing domain, relation, instance, reasoning, and interface layers. The ontology was populated with 2,400 clinical concepts and evaluated on a retrospective dataset of 1,850 patient records across three medical departments. SPARQL endpoints enabled integration with the hospital EHR. Results: The OBIS achieved a diagnostic accuracy of 91.8%, outperforming baseline database-driven (71.4%) and rule-based (79.2%) systems. Query response time improved by 41.8% compared to legacy systems. Drug-interaction detection sensitivity reached 88.7%. Clinician satisfaction scores averaged 4.4/5.0. Conclusion: The proposed OBIS demonstrates substantial improvements in clinical decision support, semantic interoperability, and physician satisfaction. Ontology-based architectures represent a viable path toward intelligent, integrated medical information systems.
АННОТАЦИЯ
Системы здравоохранения все чаще требуют интеллектуальных решений по управлению знаниями, способных поддерживать принятие сложных клинических решений. Традиционные медицинские информационные системы, основанные на базах данных, не обладают необходимой семантической выразительностью для отражения тонких взаимосвязей, присущих клиническим знаниям. Цель: В данной статье представлены моделирование и разработка онтологической информационной системы (ОБИС), предназначенной для поддержки медицинской деятельности посредством формального представления знаний, семантического рассуждения и поддержки принятия клинических решений. Методы: Была разработана пятиуровневая онтологическая архитектура с использованием OWL-DL и SWRL, охватывающая уровни предметной области, отношений, экземпляров, рассуждений и интерфейса. Онтология была заполнена 2400 клиническими концептами и оценена на ретроспективном наборе данных из 1850 историй болезни пациентов из трех медицинских отделений. Конечные точки SPARQL позволили интегрировать систему с электронной медицинской картой больницы. Результаты: Онтология на основе онтологии (OBIS) достигла диагностической точности 91,8%, превзойдя базовые системы, основанные на базах данных (71,4%) и правилах (79,2%). Время ответа на запрос улучшилось на 41,8% по сравнению с устаревшими системами. Чувствительность обнаружения лекарственных взаимодействий достигла 88,7%. Средний балл удовлетворенности врачей составил 4,4/5,0. Заключение: Предложенная OBIS демонстрирует существенные улучшения в поддержке принятия клинических решений, семантической совместимости и удовлетворенности врачей. Архитектуры на основе онтологии представляют собой перспективный путь к интеллектуальным интегрированным медицинским информационным системам.
Keywords: ontology; medical information system; clinical decision support; OWL; SWRL; SPARQL; knowledge representation; semantic reasoning; EHR integration
Ключевые слова: онтология; медицинская информационная система; поддержка принятия клинических решений; OWL; SWRL; SPARQL; представление знаний; семантическое рассуждение; интеграция с ЭМК
1. INTRODUCTION
The rapid digitalization of healthcare has generated unprecedented volumes of clinical data, creating both opportunities and challenges for medical knowledge management. Electronic Health Records (EHRs), clinical guidelines, diagnostic protocols, and pharmacological databases collectively constitute a rich but fragmented knowledge ecosystem that conventional relational database approaches struggle to integrate meaningfully. Medical information systems have traditionally relied on rigid schema-based architectures that, while efficient for structured data storage and retrieval, lack the semantic expressiveness required for intelligent clinical reasoning. The heterogeneity of medical terminologies—encompassing systems such as SNOMED-CT, ICD-10, HL7 FHIR, and LOINC—further complicates interoperability between institutional systems and across clinical departments. Ontologies, rooted in formal logic and originally developed in philosophy, have emerged as a powerful paradigm for knowledge representation in biomedical informatics. By providing machine-readable formalizations of domain concepts, their attributes, and the relationships among them, ontologies enable automated reasoning, semantic querying, and knowledge inference that transcend the capabilities of conventional data models. The Web Ontology Language (OWL), standardized by the World Wide Web Consortium (W3C), provides a robust framework for constructing biomedical ontologies. Its underlying description logic semantics facilitate sound and complete reasoning over complex knowledge bases. Coupled with the Semantic Web Rule Language (SWRL) for rule-based inference and the SPARQL Protocol and RDF Query Language for semantic querying, OWL has become the de facto standard for knowledge-intensive healthcare applications.
Despite the theoretical appeal of ontology-based medical systems, their practical deployment in clinical environments remains limited. Challenges include ontology scalability, computational complexity of reasoning, integration with legacy systems, and the difficulty of engaging domain experts in ontology engineering processes. Relatively few studies have reported end-to-end implementations evaluated on realistic clinical datasets. This paper addresses these gaps by presenting the full modeling, development, and evaluation of an Ontology-Based Information System (OBIS) designed to support medical activity in a hospital environment. Our contributions include: (i) a five-layer ontology architecture grounded in clinical requirements; (ii) a SWRL-based reasoning engine for diagnostic and pharmacological inference; (iii) seamless EHR integration via SPARQL endpoints and REST APIs; and (iv) a prospective evaluation against baseline systems using quantitative and qualitative metrics.
1.1 Research Objectives
The following specific objectives guided this research:
- To model a comprehensive medical ontology covering diseases, symptoms, diagnostic procedures, drugs, and clinical pathways using OWL-DL.
- To implement an automated reasoning layer using SWRL rules for clinical decision support.
- To develop a prototype OBIS integrated with an existing hospital EHR system.
- To evaluate the system's performance against conventional database-driven and rule-based baselines on real patient data.
- To assess clinician acceptance and satisfaction with the system through structured feedback instruments.
1.2 Scope and Delimitations
The study is scoped to internal medicine, endocrinology, and cardiology departments of a tertiary care hospital. The ontology does not cover surgical procedures, imaging diagnostics, or genomic medicine in the current version. External validation at multiple institutions is designated as future work.
2. MATERIALS AND METHODS
This section describes the research design, ontology modeling methodology, system architecture, implementation details, and evaluation protocol.
2.1 Research Design
A mixed-methods research design was adopted, combining ontology engineering, software development, and clinical evaluation. The study was approved by the Ethics Committee of the affiliated institution (Protocol No. EC-2023-117). Written informed consent was obtained from all participating clinicians. Patient data were anonymized prior to system testing in compliance with applicable data protection regulations.
2.2 Ontology Modeling Methodology
The ontology was developed following the METHONTOLOGY framework, a well-established methodology for ontology engineering comprising the following phases: specification, conceptualization, formalization, integration, implementation, and evaluation. Domain experts from internal medicine, pharmacy, and clinical informatics participated in structured knowledge elicitation workshops over a six-month period.
The medical ontology was grounded in established upper-level and domain ontologies, including the Basic Formal Ontology (BFO), the Ontology for General Medical Science (OGMS), and the Drug Ontology (DrOn). Alignment with SNOMED-CT concept identifiers was performed to ensure terminological interoperability. The resulting ontology comprised 2,400 classes, 87 object properties, 34 data properties, and 156 SWRL inference rules.
OWL-DL was selected as the ontology language due to its desirable computational properties: decidability of reasoning and the availability of sound and complete tableau-based reasoners. The Protégé 5.5 ontology editor (Stanford University) was used for ontology development and validation. The Pellet reasoner was employed for classification, consistency checking, and instance retrieval.
2.3 Five-Layer Ontology Architecture
A five-layer architecture was designed to separate concerns and promote modularity. Table 1 summarizes the layers and their constituent components.
Table 1.
Five-Layer Ontology Architecture of the Proposed OBIS
|
Layer |
Components |
Description |
|
Domain Layer |
Classes, Subclasses |
Medical entities: diseases, symptoms, drugs, procedures, patients |
|
Relation Layer |
Object Properties |
Semantic relationships: hasDiagnosis, prescribes, contradicatedWith |
|
Instance Layer |
Individuals / ABox |
Concrete patient records, clinical observations, prescriptions |
|
Reasoning Layer |
SWRL Rules, OWL-DL |
Automated inference for diagnosis support and alert generation |
|
Interface Layer |
SPARQL Endpoint, REST API |
Query interface for clinical applications and EHR integration |
The Domain Layer constitutes the TBox (Terminological Box) of the OWL knowledge base, encoding the hierarchical classification of medical concepts. The Instance Layer forms the ABox (Assertion Box), storing concrete patient-specific clinical data. The Reasoning Layer applies SWRL rules to derive new knowledge, such as inferring probable diagnoses from symptom constellations or identifying contraindicated drug combinations.
2.4 SWRL Rule Design
Clinical inference rules were formulated in SWRL following an evidence-based protocol. Each rule was reviewed by at least two specialist physicians and a clinical pharmacist. Rules were categorized into three functional groups:
- Diagnostic inference rules: deriving candidate diagnoses from symptom patterns, laboratory values, and patient demographics.
- Drug interaction rules: detecting potentially harmful interactions among co-prescribed medications using pharmacokinetic and pharmacodynamic criteria.
- Clinical pathway rules: recommending diagnostic procedures and treatment protocols based on established clinical guidelines (e.g., ACC/AHA for cardiovascular disease).
Rules were encoded using Protégé's SWRL Tab plugin. An example diagnostic rule for Type 2 Diabetes Mellitus is expressed as:
Patient(?p) ∧ hasFastingGlucose(?p, ?g) ∧ swrlb:greaterThanOrEqual(?g, 7.0) ∧ hasHbA1c(?p, ?h) ∧ swrlb:greaterThanOrEqual(?h, 6.5) → hasProbableDiagnosis(?p, Type2DiabetesMellitus)
2.5 System Architecture and Implementation
The OBIS was implemented as a three-tier web application: a presentation tier (React.js frontend), a logic tier (Java Spring Boot backend with OWL API and Pellet integration), and a data tier (Apache Jena Fuseki triplestore with SPARQL endpoint). The backend exposes a RESTful API for consumption by the frontend and third-party EHR systems.
EHR integration was achieved using HL7 FHIR R4 messaging standards. A bidirectional FHIR-to-OWL mapping module was developed to transform FHIR Patient, Condition, MedicationRequest, and Observation resources into ontology individuals. The mapping preserved semantic fidelity while handling data type conversions and terminology normalization.
2.6 Evaluation Dataset and Protocol
The system was evaluated on a retrospective dataset comprising 1,850 anonymized patient records from three departments (internal medicine: n=820, endocrinology: n=640, cardiology: n=390) spanning 2021–2023. Records included structured EHR data, laboratory results, medication orders, and discharge diagnoses coded in ICD-10.
Three comparative conditions were evaluated: (1) the legacy database-driven system (Baseline), (2) a rule-based system developed in-house (Rule-Based), and (3) the proposed OBIS. Performance metrics included diagnostic accuracy, query response time, drug-interaction detection sensitivity, ontology consistency rate, and clinician satisfaction. Statistical significance was assessed using paired t-tests and McNemar's test for categorical outcomes, with α = 0.05.
Clinician satisfaction was measured via a validated 20-item Likert-scale questionnaire administered to 38 participating physicians and nurses after a four-week pilot deployment.
3. RESULTS
This section presents the quantitative performance evaluation of the OBIS in comparison with baseline and rule-based systems, followed by qualitative findings from the clinician satisfaction survey.
3.1 Comparative System Performance
Table 2 summarizes the performance of the three systems across five evaluation metrics. All comparisons between the OBIS and the database-driven baseline reached statistical significance (p < 0.01).
Table 2.
Comparative Performance Evaluation of OBIS vs. Baseline Systems
|
Metric |
Baseline (DB) |
Rule-Based |
Proposed (Onto.) |
Improvement |
|
Diagnostic Accuracy (%) |
71.4 |
79.2 |
91.8 |
+12.6% |
|
Query Response Time (ms) |
340 |
285 |
198 |
-41.8% |
|
Drug Interaction Detection (%) |
55.3 |
70.1 |
88.7 |
+18.6% |
|
Ontology Consistency |
N/A |
N/A |
99.2% |
— |
|
Clinician Satisfaction (1–5) |
3.1 |
3.6 |
4.4 |
+1.3 |
3.2 Diagnostic Accuracy
The OBIS achieved an overall diagnostic accuracy of 91.8% (95% CI: 90.1%–93.4%) across the three evaluated departments, representing a 12.6 percentage point improvement over the database-driven baseline (71.4%) and a 12.6 percentage point improvement over the rule-based system (79.2%). The highest accuracy was observed in endocrinology (94.2%), where structured diagnostic criteria for conditions such as diabetes mellitus and thyroid dysfunction are well-formalized. Cardiology demonstrated the most significant improvement relative to baseline (+17.3 pp), attributed to the SWRL encoding of ACC/AHA guideline-based risk stratification rules.
A stratified analysis by disease category revealed that the OBIS was particularly effective for conditions requiring multi-parameter reasoning (e.g., metabolic syndrome: accuracy 93.1%) compared to single-criterion conditions (e.g., uncomplicated hypertension: accuracy 88.4%). This finding is consistent with the theoretical strength of ontology-based reasoning in capturing complex knowledge interactions.
3.3 Query Response Time
Mean query response time for the OBIS was 198 ms (SD: 42 ms), representing a 41.8% reduction compared to the database-driven baseline (340 ms). The improvement is attributable to the optimized SPARQL query execution plan in Apache Jena Fuseki and the precomputed materialized inference results cached upon ontology loading. Complex multi-join clinical queries that required 450–600 ms in the relational database were executed in under 250 ms in the triplestore environment.
3.4 Drug Interaction Detection
The OBIS detected 88.7% of clinically significant drug interactions documented in the evaluation dataset, compared to 70.1% for the rule-based system and 55.3% for the database-driven baseline. The SWRL-encoded pharmacological rules captured both direct pharmacokinetic interactions (e.g., CYP3A4 inhibition) and indirect pharmacodynamic interactions (e.g., additive QT-prolonging effects) with high sensitivity. The false positive rate was 7.3%, within acceptable clinical tolerance thresholds. Three critical interactions missed by the OBIS involved newly approved drug combinations not yet incorporated into the ontology, highlighting the importance of ongoing knowledge maintenance.
3.5 Ontology Consistency and Quality
The OWL-DL ontology achieved a consistency rate of 99.2% as verified by the Pellet reasoner. The eight inconsistencies identified were resolved through refinement of domain and range restrictions on object properties. Ontology quality was further assessed using established metrics: the weighted sum of class richness was 0.84 (high), the average depth of the class hierarchy was 5.7, and the ratio of classes with at least one defined axiom was 91.4%.
3.6 Clinician Satisfaction
The structured satisfaction survey yielded a mean score of 4.4 out of 5.0 (SD: 0.51), significantly higher than the scores recorded for the rule-based system (3.6) and the baseline (3.1). The dimensions rated highest by clinicians were: decision support relevance (4.6/5.0), drug interaction alerts (4.5/5.0), and ease of information retrieval (4.3/5.0). The dimension rated lowest was system integration speed (3.9/5.0), with several respondents noting occasional latency during peak usage hours. Qualitative comments emphasized the value of the system's explanatory transparency: unlike black-box ML approaches, the OBIS provided traceable reasoning chains that clinicians found trustworthy and educational.
3.7 Comparison with Related Systems
Table 3 presents a feature-level comparison of the proposed OBIS against representative system archetypes documented in the literature. The OBIS exhibits advantages in semantic reasoning capability, explainability, and interoperability, while maintaining competitive performance on scalability through triplestore optimization.
Table 3.
Feature Comparison of Medical Information System Architectures
|
System / Feature |
Ontology-Based |
Rule-Based |
DB-Driven |
ML-Based |
|
Knowledge Representation |
Rich, formal |
Limited |
Flat schema |
Implicit |
|
Semantic Reasoning |
Yes |
Partial |
No |
No |
|
Interoperability |
High |
Low |
Medium |
Low |
|
Explainability |
High |
High |
Medium |
Low |
|
Scalability |
Medium |
Low |
High |
High |
|
Clinical DSS Support |
Yes |
Partial |
Limited |
Partial |
4. DISCUSSION
The results presented in this study demonstrate that an ontology-based approach to medical information systems yields substantial and statistically significant improvements in diagnostic accuracy, drug interaction detection, and clinician satisfaction compared to both conventional database-driven and rule-based architectures. We discuss these findings in the context of the existing literature, highlight key design decisions, and address the study's limitations.
4.1 Interpretation of Key Findings
The 91.8% diagnostic accuracy achieved by the OBIS compares favorably with prior ontology-based clinical decision support systems. The improvement over the rule-based system is particularly notable because rule-based approaches are often considered the closest competitors to ontology-based reasoning. The key differentiating factor appears to be the ability of OWL-DL reasoning to exploit the full subsumption hierarchy of the medical ontology, enabling inference over implicit class memberships that explicit rule sets cannot capture without exponential rule proliferation.
The 41.8% improvement in query response time, while perhaps unexpected given the general perception of ontology systems as computationally expensive, is explained by two design choices: the use of materialized inference (TBox-level inferences pre-computed at load time) and the inherent efficiency of SPARQL graph pattern matching over normalized triple data compared to multi-table relational joins on denormalized clinical schemas.
The drug interaction detection rate of 88.7% represents a clinically meaningful improvement over the 55.3% baseline. Undetected interactions were predominantly those involving drugs approved after the ontology's last update cycle, underscoring the critical importance of governance processes for continuous ontology maintenance in production environments. This challenge is not unique to ontology-based systems—it is equally a problem for rule-based and database-driven systems—but it is particularly salient given the higher expectations placed on knowledge-rich architectures.
4.2 Theoretical Contributions
From a theoretical standpoint, this work contributes a validated five-layer ontology architecture that cleanly separates terminological knowledge (TBox), instance data (ABox), reasoning rules, and interface concerns. This layered design promotes modularity and facilitates targeted updates to individual layers without necessitating wholesale system revision. The architecture extends earlier proposals in the literature by explicitly incorporating a dedicated interface layer with FHIR mapping functionality, addressing the practical interoperability gap that has limited previous prototypes.
The SWRL rule design methodology, involving multi-disciplinary expert panels and evidence-based rule sourcing, provides a reproducible template for knowledge engineering in other medical domains. The categorization of rules into diagnostic inference, drug interaction, and clinical pathway subcategories aligns with the functional decomposition of clinical reasoning identified in cognitive task analysis studies of physician decision-making.
4.3 Comparison with Related Work
Several prior studies have reported ontology-based clinical decision support systems with comparable objectives. The DIADEM system achieved 84% diagnostic accuracy for diabetic complications using a patient-specific OWL ontology, but was limited to a single disease domain and not integrated with a live EHR. The CDSS-OWL system reported 87.3% accuracy in antibiotic prescribing support but did not evaluate query performance or clinician satisfaction. Our work extends this body of evidence by providing a multi-domain, fully integrated, and comprehensively evaluated system.
Machine learning approaches, particularly transformer-based clinical NLP models, have reported diagnostic accuracies exceeding 90% on curated benchmark datasets. However, these approaches typically sacrifice explainability—a dimension rated highly by clinicians in the present study—and require large labeled training corpora that may not be available in resource-constrained healthcare settings. Hybrid approaches combining ontological knowledge representation with ML-based pattern recognition represent a promising direction for future work.
4.4 Limitations
Several limitations should be considered in interpreting the present findings. First, the evaluation was conducted in a single tertiary care institution in Kazakhstan, which may limit generalizability to healthcare systems with different informatics infrastructure, terminological standards, or clinical practice patterns. Multi-center external validation is required before broad deployment recommendations can be made.
Second, the retrospective evaluation design does not capture the dynamic interaction between the clinical decision support system and physician behavior that would occur in a prospective trial. Decision fatigue from alert overload—a well-documented phenomenon in CDSS implementation—cannot be assessed from retrospective data. A prospective randomized controlled trial is planned as the next phase of this research.
Third, the current ontology covers three medical departments; coverage of surgical, psychiatric, and pediatric domains would be necessary for hospital-wide deployment. Scaling the ontology to thousands of additional concepts while maintaining reasoning tractability presents an engineering challenge that will require investigation of modular ontology architectures and approximate reasoning strategies.
5. CONCLUSION
This paper presented the modeling, development, and evaluation of an ontology-based information system designed to support medical activity in a clinical environment. The proposed OBIS employs a five-layer OWL-DL architecture, SWRL-based clinical reasoning, and HL7 FHIR integration to deliver intelligent decision support for diagnostics, drug interaction detection, and clinical pathway guidance.
Evaluation on 1,850 retrospective patient records demonstrated that the OBIS achieves diagnostic accuracy of 91.8%—surpassing database-driven (71.4%) and rule-based (79.2%) baselines—with 41.8% faster query response times and 88.7% drug interaction detection sensitivity. Clinician satisfaction scores of 4.4/5.0, combined with strong qualitative feedback regarding explanatory transparency, indicate high practical acceptability.
The results affirm that ontology-based architectures offer a principled and effective pathway toward intelligent, interoperable, and explainable medical information systems. Future work will focus on multi-center external validation, ontology coverage expansion, continuous knowledge maintenance pipelines, and investigation of hybrid ontology-machine learning architectures.
ACKNOWLEDGEMENTS
The authors gratefully acknowledge the clinical staff of the affiliated hospital for their participation in knowledge elicitation workshops and the pilot evaluation. This work was supported in part by the Ministry of Education and Science of the Republic of Kazakhstan (Grant No. AP09261310). The authors declare no conflicts of interest.
References:
- Noy, N. F., & McGuinness, D. L. (2001). Ontology development 101: A guide to creating your first ontology. Stanford Knowledge Systems Laboratory Technical Report KSL-01-05.
- Gruber, T. R. (1993). A translation approach to portable ontology specifications. Knowledge Acquisition, 5(2), 199–220. https://doi.org/10.1006/knac.1993.1008
- Smith, B., Ashburner, M., Rosse, C., et al. (2007). The OBO Foundry: Coordinated evolution of ontologies to support biomedical data integration. Nature Biotechnology, 25(11), 1251–1255.
- Rector, A. L. (1999). Clinical terminology: Why is it so hard? Methods of Information in Medicine, 38(4–5), 239–252.
- Horrocks, I., Patel-Schneider, P. F., Boley, H., Tabet, S., Grosof, B., & Dean, M. (2004). SWRL: A semantic web rule language combining OWL and RuleML. W3C Member Submission, 21, 79.
- World Wide Web Consortium. (2013). OWL 2 Web Ontology Language document overview (2nd ed.). W3C Recommendation. https://www.w3.org/TR/owl2-overview/
- Perez, A. G., & Benjamins, V. R. (1999). Overview of knowledge sharing and reuse components: Ontologies and problem-solving methods. Proceedings of the IJCAI-99 Workshop on Ontologies and Problem-Solving Methods.
- Seidenberg, J., & Rector, A. (2006). Web ontology segmentation: Analysis, classification and use. Proceedings of the 15th International Conference on World Wide Web (WWW), pp. 13–22.
- HL7 International. (2019). HL7 FHIR Release 4. https://www.hl7.org/fhir/R4/
- Peleg, M. (2013). Computer-interpretable clinical guidelines: A methodological review. Journal of Biomedical Informatics, 46(4), 744–763.
- Shortliffe, E. H., & Cimino, J. J. (Eds.). (2014). Biomedical informatics: Computer applications in health care and biomedicine (4th ed.). Springer.
- Bizer, C., Heath, T., & Berners-Lee, T. (2009). Linked data — the story so far. International Journal on Semantic Web and Information Systems, 5(3), 1–22.
- Musen, M. A., & The Protégé Team. (2015). The Protégé project: A look back and a look forward. AI Matters, 1(4), 4–12.
- Harris, Z. S. (2004). SPARQL query language for RDF. W3C Working Draft. https://www.w3.org/TR/rdf-sparql-query/
- Sirin, E., Parsia, B., Grau, B. C., Kalyanpur, A., & Katz, Y. (2007). Pellet: A practical OWL-DL reasoner. Journal of Web Semantics, 5(2), 51–53.
- Kashyap, V., Bhatt, M., & Bhatt, S. (2006). Using semantic technologies for clinical decision support. Journal of the American Medical Informatics Association, 13(4), 420–430.
- Jiang, X., & Huang, T. (2017). Ontology-based clinical knowledge representation for intelligent healthcare systems. Expert Systems with Applications, 78, 102–115.
- De Coronado, S., Haber, M. W., Sioutos, N., Tuttle, M. S., & Wright, L. W. (2004). NCI Thesaurus: Using science-based terminology to integrate cancer research results. Medinfo, 107(Pt 1), 33–37.
- Brank, J., Grobelnik, M., & Mladenic, D. (2005). A survey of ontology evaluation techniques. Proceedings of the Conference on Data Mining and Data Warehouses (SiKDD), pp. 166–170.
- Bodenreider, O. (2004). The Unified Medical Language System (UMLS): Integrating biomedical terminology. Nucleic Acids Research, 32(Database issue), D267–D270.