{ "cells": [ { "cell_type": "markdown", "metadata": {}, "source": [ "# Analyzing healthcare FHIR data with Amazon Neptune" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "This Jupyter Notebook extends the walkthrough described in the blog on [Analyzing healthcare FHIR data with Amazon Neptune](https://aws.amazon.com/blogs/database/analyze-healthcare-fhir-data-with-amazon-neptune/). Go through the set up steps 1-3 described in the blog before issuing the queries below." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Prerequisite: Load data from S3 into Amazon Neptune via bulk loader" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Executing the cell below will open a form that you can use to submit a bulk load request to Neptune.\n", "\n", "Adapt the values as follows:\n", "- Source: Provide the name of your bucket as Source (e.g. s3://example.com/). Pay attention to the trailing slash.\n", "- Format: Select __turtle__ from the dropdown.\n", "- Region: Correct the region if it doesn't reflect the region in which your Amazon S3 bucket and Amazon Neptune cluster was created.\n", "- Load ARN: Provide the ARN of the IAM user you created.\n", "- Parallelism: Select __OVERSUBSCRIBE__\n", "\n", "Keep the default values for the remaining properties. Submit load request.\n", "\n", "Depending on the instance size of your database instance, it can take some time until the operation completes." ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "%load" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "If you restarted your kernel before the load was completed, you can check the status of the load by executing the following two cells." ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "# Get ID of submitted request\n", "%load_ids" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "# Get status of request by ID\n", "%load_status " ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Basic introduction to SPARQL" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "SPARQL is a query language for the Resource Description Framework (RDF), which is a graph data format designed for the web. Amazon Neptune is compatible with SPARQL 1.1. This means that you can connect to a Neptune DB instance and query the graph using the query language as described in the [SPARQL 1.1 Query specification.](https://www.w3.org/TR/sparql11-query/).\n", "\n", "A query in SPARQL consists of a SELECT clause to specify the variables to return and a WHERE clause to specify which data to match in the graph. If you are unfamiliar with SPARQL queries, see [Writing Simple Queries](https://www.w3.org/TR/sparql11-query/#WritingSimpleQueries) in the SPARQL 1.1 Query Language.\n", "\n", "The following query retrieves ten random triples from your graph. Triples are statements consiting of subject, predicate, object." ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "%%sparql --expand-all\n", "\n", "SELECT *\n", "WHERE\n", "{ ?s ?p ?o . }\n", "LIMIT 10" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "You can specify which triples you want to retrieve by specifiying subject, predicate, and/or object. In the example below we introduce a variable for the subject. The query retrieves all triples of a variable subject that is related to the object http://hl7.org/fhir/QuestionnaireResponse via the predicate http://www.w3.org/1999/02/22-rdf-syntax-ns#type. This triple matches subjects that are of the type QuestionnaireResponse. Instead of returning all values, we only return the values of the subjects. In this case ten questionnaire response IDs." ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "%%sparql --expand-all\n", "\n", "SELECT ?questionnaireResponse\n", "WHERE\n", "{ \n", " ?questionnaireResponse .\n", "}\n", "LIMIT 10" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "For better readability, we introduce two Prefixes, __fhir__ and __rdf__, that can be used in the WHERE clause." ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "%%sparql --expand-all\n", "\n", "PREFIX fhir: \n", "PREFIX rdf: \n", "\n", "SELECT ?questionnaireResponse\n", "WHERE\n", "{ \n", " ?questionnaireResponse rdf:type fhir:QuestionnaireResponse .\n", "}\n", "LIMIT 10" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Instead of SELECT you can use CONSTRUCT to return a new RDF graph. You can specify the format of this graph in the CONSTRUCT section. Also, you can use slahes to combine multiple predicates that should be followed by the query.\n", "The query below constructs a new graph based on the information of patient to questionnaire responses mapping. \n", "\n", "Navigate to the __Graph__ tab to view the graph visualization." ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "%%sparql --expand-all\n", "\n", "PREFIX fhir: \n", "PREFIX qr: \n", "\n", "CONSTRUCT { \n", " ?questionnaireResponse fhir:value ?patient .\n", " ?questionnaireResponse a fhir:QuestionnaireResponse .\n", " ?patient a fhir:Patient .\n", "}\n", "WHERE { \n", " ?questionnaireResponse qr:subject/fhir:Reference.reference/fhir:value ?patient .\n", "}" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Sample Queries" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### 1. Identify patients, that work(ed) in same industry" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "The first query matches questionnaire responses with the same answer to question 4.2 “Employer industry” and returns the patients that correspond to these. As a result you can quickly identify the clusters of patients, that work(ed) in the same industry. The visualization makes it easy to identify industries that were named very often or less frequently. Pharma & Health represents an industry that was named by a large number of patients.\n", "\n", "__Hint:__ Drag the nodes in the graph visualization to separate the clusters from each other for better visibility." ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "%%sparql --expand-all\n", "\n", "PREFIX fhir: \n", "PREFIX qr: \n", "\n", "CONSTRUCT {\n", " ?questionnaireResponse fhir:value ?patient ;\n", " fhir:value ?industryAnswer .\n", " \n", " ?questionnaireResponse a fhir:QuestionnaireResponse .\n", " ?patient a fhir:Patient .\n", "}\n", "WHERE {\n", " ?questionnaireResponse qr:subject/fhir:Reference.reference/fhir:value ?patient ;\n", " qr:item/qr:item.item ?item4_2 .\n", " ?item4_2 qr:item.item.answer/qr:item.item.answer.valueString/fhir:value ?industryAnswer ;\n", " qr:item.item.linkId/fhir:value \"4.2\" .\n", "}\n" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### 2. Identify industries with common hazards" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "This query matches the answers to question 4.2 “Employer industry” and 4.3 “Hazards in Workplace”. Answers stating no hazards are filtered out. This gives an overview of hazards that ore more common in some industries than in others.\n", "\n", "Given the density of nodes, you can identify two general clusters of industries related to more and less threatening hazards.\n", "The first cluster contains industries related to safety, biological, chemical, and physical hazards. \n", "\n", "The construction industry is for example closely related to safety hazards. The second cluster contains industries related to ergonomic and workload hazards. The service & crafts industry is for example, linked to ergonomic hazards. Some questionnaire responses link industries with hazards from the other cluster, but most patients answered on hazards within one cluster. You can use this information to dive deeper into these cases and understand where the difference comes from.\n", "\n", "__Hint:__ Drag the nodes in the graph visualization to separate the clusters from each other for better visibility." ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "%%sparql --expand-all\n", "\n", "PREFIX fhir: \n", "PREFIX qr: \n", "\n", "\n", "CONSTRUCT {\n", " ?parentItem4 fhir:value ?industryAnswer ;\n", " fhir:value ?hazardAnswer .\n", "\n", " ?parentItem4 a qr:item.item .\n", " ?industryAnswer a fhir:value .\n", "}\n", "WHERE {\n", " ?industryAnswer ^fhir:value/^qr:item.item.answer.valueString/^qr:item.item.answer ?item4_2 .\n", " ?item4_2 qr:item.item.linkId/fhir:value \"4.2\" ;\n", " ^qr:item.item ?parentItem4 .\n", " ?parentItem4 qr:item.item ?item4_3 .\n", " ?item4_3 qr:item.item.linkId/fhir:value \"4.3\" ;\n", " qr:item.item.answer/qr:item.item.answer.valueString/fhir:value ?hazardAnswer .\n", " \n", " FILTER('None' != ?hazardAnswer)\n", "}" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### 3. Get questionnaires with similar answers for question group compared to single questionnaire" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "This sample query compares answers of patients in the question section 1 \"Drinking and smoking behavior”, which contains five questions:\n", "\n", "1. How many liters of beer do you consume in a week? \n", "2. How many liters of wine do you consume in a month? \n", "3. How many years have you been smoking?\n", "4. How many cigarettes do you currently smoke per day? \n", "5. How many cigars do you currently smoke per week?\n", "\n", "The result of the query is a list of questionnaire responses that matches the answers of a particular questionnaire response (QuestionnaireResponse/92d290e2-26a6-4474-9085-71f3b146dfd5) in at least 4 of 5 answers. The questionnaire response against which the responses are matched against is also included in the result list in this example." ] }, { "cell_type": "code", "execution_count": null, "metadata": { "scrolled": false }, "outputs": [], "source": [ "%%sparql\n", "\n", "PREFIX fhir: \n", "PREFIX qr: \n", "\n", "SELECT ?similarQR (count(?sameAnswerValue) as ?sameAnswerCount) \n", "WHERE {\n", " qr:item ?parentItem1_a .\n", " ?parentItem1_a qr:item.linkId/fhir:value \"1\" ;\n", " qr:item.item ?subItem_a .\n", " ?subItem_a qr:item.item.answer/qr:item.item.answer.valueInteger/fhir:value ?sameAnswerValue ;\n", " qr:item.item.text/fhir:value ?question .\n", " \n", " ?similarQR qr:item ?parentItem1_b .\n", " ?parentItem1_b qr:item.linkId/fhir:value \"1\" ;\n", " qr:item.item ?subItem_b .\n", " ?subItem_b qr:item.item.answer/qr:item.item.answer.valueInteger/fhir:value ?sameAnswerValue ;\n", " qr:item.item.text/fhir:value ?question .\n", "}\n", "GROUP BY ?similarQR\n", "HAVING (?sameAnswerCount > 3) \n", "ORDER BY DESC(?sameAnswerCount)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Conclusion" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "This notebook showed you how easy it is to load data into a graph. You also issued three different queries to illustrate how you can generate insights from FHIR data.\n", "\n", "To dive deeper into the topic see the [Amazon Neptune developer resources](https://aws.amazon.com/neptune/developer-resources/) for documentation links, other blog posts, videos, and sample code repositories." ] } ], "metadata": { "kernelspec": { "display_name": "Python 3", "language": "python", "name": "python3" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 3 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.7.12" }, "vscode": { "interpreter": { "hash": "aee8b7b246df8f9039afb4144a1f6fd8d2ca17a180786b69acc140d282b71a49" } } }, "nbformat": 4, "nbformat_minor": 2 }