diff --git a/.dockerignore b/.dockerignore index 0c6d25db4..bd235c830 100644 --- a/.dockerignore +++ b/.dockerignore @@ -10,6 +10,6 @@ node_modules coverage # Ignore documentation related files -additionaldocs +docs images *.md diff --git a/.kiro/steering/documentation.md b/.kiro/steering/documentation.md index 6b23caa1c..f0c6b0079 100644 --- a/.kiro/steering/documentation.md +++ b/.kiro/steering/documentation.md @@ -25,7 +25,7 @@ ### Documentation Hierarchy - **README.md**: Project overview and quick start -- **additionaldocs/**: Detailed guides organized by user journey +- **docs/**: Detailed guides organized by user journey - `getting-started/`: Installation, first connection, basic usage - `user-guide/`: Feature documentation for end users - `deployment/`: Production deployment guides @@ -62,7 +62,7 @@ ``` README.md # Project overview and quick start -additionaldocs/ +docs/ ├── getting-started/ │ ├── installation.md # Local and Docker setup │ ├── first-connection.md # Connecting to your first database @@ -92,6 +92,44 @@ additionaldocs/ - Use descriptive names that match user tasks: `first-connection.md` - Group related content in folders by user journey +### Folder README Files + +Each documentation folder should include a `README.md` file that serves as a +table of contents for that folder: + +- **Purpose**: Explain what the folder contains and who it's for +- **Summary**: Briefly describe the folder's purpose and content +- **File Listing**: List all files in the folder with brief descriptions +- **Reading Order**: Suggest the recommended order for reading files +- **Prerequisites**: Note any required knowledge or setup before starting +- **Navigation**: Link to parent folder and related sections + +Example folder README structure: + +```markdown +# Getting Started + +This section helps new users install and configure Graph Explorer for the first +time. + +## Files in this section + +1. [Installation](installation.md) - Local and Docker setup options +2. [First Connection](first-connection.md) - Connect to your first database +3. [Basic Usage](basic-usage.md) - Essential features walkthrough + +## Prerequisites + +- Basic familiarity with graph databases +- Access to a graph database endpoint +- Docker or Node.js >=24.4.0 installed + +## Next Steps + +After completing this section, see the [User Guide](../user-guide/) for detailed +feature documentation. +``` + ## Formatting Standards ### Code Examples diff --git a/.kiro/steering/structure.md b/.kiro/steering/structure.md index ecae787e8..ad4e16764 100644 --- a/.kiro/steering/structure.md +++ b/.kiro/steering/structure.md @@ -14,7 +14,7 @@ graph-explorer/ │ ├── graph-explorer/ # React frontend application │ ├── graph-explorer-proxy-server/ # Express.js backend server │ └── shared/ # Shared types and utilities -├── additionaldocs/ # Documentation +├── docs/ # Documentation ├── samples/ # Example configurations └── [root config files] # Workspace-level configuration ``` diff --git a/Changelog.md b/Changelog.md index a840819c2..bb33dcf4e 100644 --- a/Changelog.md +++ b/Changelog.md @@ -137,7 +137,7 @@ editor with Neptune Analytics for the first time. As before, if you are a Neptune user concerned about data integrity, check out the -[instructions on configuring IAM permissions to restrict mutations](./README.md#permissions), +[instructions on configuring IAM permissions to restrict mutations](./docs/user-guide/security.md#permissions), ensuring that users can enforce read-only operations. ### New features & major changes @@ -163,7 +163,7 @@ ensuring that users can enforce read-only operations. Graph Explorer v2.0 marks a significant milestone with the introduction of the new -[query editor for Gremlin connections](./additionaldocs/features/README.md#query-search). +[query editor for Gremlin connections](./docs/user-guide/graph-explorer.md#query-search). This powerful feature allows users to enter any valid Gremlin query and visualize the returned nodes, edges, and scalar values directly from the database. You can review the results in the sidebar and choose to add all nodes @@ -171,7 +171,7 @@ database. You can review the results in the sidebar and choose to add all nodes For Neptune users concerned about data integrity, [the README](./README.md) now includes -[instructions on configuring IAM permissions to restrict mutations](./README.md#permissions), +[instructions on configuring IAM permissions to restrict mutations](./docs/user-guide/security.md#permissions), ensuring users can enforce read-only operations. The release also changes the default strategy for representing node & edge @@ -662,11 +662,11 @@ a more pleasant experience, especially for users with larger databases. - **Improved** diagnostic logging in Neptune Notebooks by adding CloudWatch logs ([#517](https://github.com/aws/graph-explorer/pull/517)) - Check out the example - [lifecycle script](additionaldocs/sagemaker/install-graph-explorer-lc.sh) + [lifecycle script](docs/deployment/aws-sagemaker/install-graph-explorer-lc.sh) and IAM policies for - [Neptune](additionaldocs/sagemaker/graph-explorer-neptune-db-policy.json) + [Neptune](docs/deployment/aws-sagemaker/graph-explorer-neptune-db-policy.json) and - [Neptune Analytics](additionaldocs/sagemaker/graph-explorer-neptune-analytics-policy.json) + [Neptune Analytics](docs/deployment/aws-sagemaker/graph-explorer-neptune-analytics-policy.json) **Bug Fixes and Minor Changes** diff --git a/README.md b/README.md index afadcef05..6b8016181 100644 --- a/README.md +++ b/README.md @@ -20,32 +20,19 @@ like [Amazon ECS](https://aws.amazon.com/ecs/). There are many ways to deploy and run Graph Explorer. If you are new to graph databases and Graph Explorer, we recommend that you check out the -[Getting Started](./additionaldocs/getting-started/README.md) guide. - -- [Local Docker Setup](./additionaldocs/getting-started/README.md#local-docker-setup) - - A quick start guide to deploying Graph Explorer locally using the official - Docker image. -- [Amazon EC2 Setup](./additionaldocs/getting-started/README.md#amazon-ec2-setup) - - A quick start guide to setting up Graph Explorer on Amazon EC2 with Neptune. -- [Local Development](./additionaldocs/getting-started/README.md#local-development-setup) - - A quick start guide building the Docker image from source code. -- [Troubleshooting](./additionaldocs/troubleshooting.md) - A collection of - helpful tips if you run in to issues while setting up Graph Explorer. +[Getting Started](./docs/getting-started/README.md) guide. + +- [Local Docker Setup](./docs/deployment/docker.md) - A quick start guide to + deploying Graph Explorer locally using the official Docker image. +- [Amazon EC2 Setup](./docs/deployment/aws-ec2.md) - A quick start guide to + setting up Graph Explorer on Amazon EC2 with Neptune. +- [Local Development](./docs/development/development-setup.md) - A quick start + guide building the Docker image from source code. +- [Troubleshooting](./docs/troubleshooting/) - A collection of helpful tips if + you run in to issues while setting up Graph Explorer. - [Samples](./samples) - A collection of Docker Compose files that show various ways to configure and use Graph Explorer. -### Minimum Recommended Versions - -Graph Explorer does not block any particular versions of graph databases, but -the queries used may or may not succeed based on the version of the query -engine. - -For Neptune databases, we recommend -[version 1.2.1.0](https://docs.aws.amazon.com/neptune/latest/userguide/engine-releases-1.2.1.0.html) -or above, which include the summary API and TinkerPop 3.6.2. - -For non-Neptune databases, we recommend at least TinkerPop 3.6. - ## Features Overview Graph Explorer offers a comprehensive set of tools for interacting with graph @@ -67,238 +54,14 @@ databases: view For complete documentation on all features and functionality, please see our -[detailed features guide](./additionaldocs/features/README.md). +[detailed user guide](./docs/user-guide/). If you're interested in our future development plans, check out our [roadmap](./ROADMAP.md) and participate in the discussions. -## Connections - -Graph Explorer supports visualizing both **property graphs** and **RDF graphs**. -You can connect to Amazon Neptune or you can also connect to open graph -databases that implement an Apache TinkerPop Gremlin endpoint or the SPARQL 1.1 -protocol, such as Blazegraph. For additional details on connecting to different -graph databases, see [Connections](./additionaldocs/connections.md). - -### Providing a Default Connection - -To provide a default connection such that initial loads of Graph Explorer always -result with the same starting connection, modify the `docker run ...` command to -either take in a JSON configuration or runtime environment variables. If you -provide both a JSON configuration and environmental variables, the JSON will be -prioritized. - -#### Environment Variables - -These are the valid environment variables used for the default connection, their -defaults, and their descriptions. - -- Required: - - `PUBLIC_OR_PROXY_ENDPOINT` - `None` - See - [Add a New Connection](#connections-ui) -- Optional - - `GRAPH_TYPE` - `None` - If not specified, multiple connections will be - created for every available graph type / query language. See - [Add a New Connection](#connections-ui) - - `USING_PROXY_SERVER` - `False` - See [Add a New Connection](#connections-ui) - - `IAM` - `False` - See [Add a New Connection](#connections-ui) - - `GRAPH_EXP_HTTPS_CONNECTION` - `True` - Controls whether Graph Explorer uses - SSL or not - - `PROXY_SERVER_HTTPS_CONNECTION` - `True` - Controls whether the server uses - SSL or not - - `GRAPH_EXP_FETCH_REQUEST_TIMEOUT` - `240000` - Controls the timeout for the - fetch request. Measured in milliseconds (i.e. 240000 is 240 seconds or 4 - minutes). - - `GRAPH_EXP_NODE_EXPANSION_LIMIT` - `None` - Controls the limit for node - counts and expansion queries. -- Conditionally Required: - - Required if `USING_PROXY_SERVER=True` - - `GRAPH_CONNECTION_URL` - `None` - See - [Add a New Connection](#connections-ui) - - Required if `USING_PROXY_SERVER=True` and `IAM=True` - - `AWS_REGION` - `None` - See [Add a New Connection](#connections-ui) - - `SERVICE_TYPE` - `neptune-db`, Set this as `neptune-db` for Neptune - database or `neptune-graph` for Neptune Analytics. - -#### JSON Configuration Approach - -First, create a `config.json` file containing values for the connection -attributes: - -```js -{ - "PUBLIC_OR_PROXY_ENDPOINT": "https://public-endpoint", - "GRAPH_CONNECTION_URL": "https://cluster-cqmizgqgrsbf.us-west-2.neptune.amazonaws.com:8182", - "USING_PROXY_SERVER": true, - "IAM": true, - "SERVICE_TYPE": "neptune-db", - "AWS_REGION": "us-west-2", - // Possible Values are "gremlin", "sparql", "openCypher" - "GRAPH_TYPE": "gremlin", - "GRAPH_EXP_HTTPS_CONNECTION": true, - "PROXY_SERVER_HTTPS_CONNECTION": true, - // Measured in milliseconds (i.e. 240000 is 240 seconds or 4 minutes) - "GRAPH_EXP_FETCH_REQUEST_TIMEOUT": 240000, - "GRAPH_EXP_NODE_EXPANSION_LIMIT": 500, -} -``` - -Pass the `config.json` file path to the `docker run` command. - -```bash -docker run -p 80:80 -p 443:443 \ - --env HOST={hostname-or-ip-address} \ - -v /path/to/config.json:/graph-explorer/config.json \ - public.ecr.aws/neptune/graph-explorer -``` - -#### Environment Variable Approach - -Provide the desired connection variables directly to the `docker run` command, -as follows: - -```bash -docker run -p 80:80 -p 443:443 \ - --env HOST={hostname-or-ip-address} \ - --env PUBLIC_OR_PROXY_ENDPOINT=https://public-endpoint \ - --env GRAPH_TYPE=gremlin \ - --env USING_PROXY_SERVER=true \ - --env IAM=false \ - --env GRAPH_CONNECTION_URL=https://cluster-cqmizgqgrsbf.us-west-2.neptune.amazonaws.com:8182 \ - --env AWS_REGION=us-west-2 \ - --env SERVICE_TYPE=neptune-db \ - --env PROXY_SERVER_HTTPS_CONNECTION=true \ - --env GRAPH_EXP_FETCH_REQUEST_TIMEOUT=240000 \ - --env GRAPH_EXP_NODE_EXPANSION_LIMIT=500 \ - public.ecr.aws/neptune/graph-explorer -``` - ## Development -For development guidance, see [Development](./additionaldocs/development.md). - -## Security - -You can use Graph Explorer to connect to a publicly accessible graph database -endpoint, or connect to a proxy endpoint that redirects to a private graph -database endpoint. - -Graph Explorer supports the HTTPS protocol by default and provides a self-signed -certificate as part of the Docker image. You can choose to use HTTP instead by -changing the -[environment variable default settings](./additionaldocs/development.md#environment-variables). - -### HTTPS Connections - -If either Graph Explorer or the proxy-server are served over an HTTPS connection -(which it is by default), you will have to bypass the warning message from the -browser due to the included certificate being a self-signed certificate. You can -bypass by manually ignoring them from the browser or downloading the correct -certificate and configuring them to be trusted. Alternatively, you can provide -your own certificate. The following instructions can be used as an example to -bypass the warnings for Chrome, but note that different browsers and operating -systems will have slightly different steps. - -1. Download the certificate directly from the browser. For example, if using - Google Chrome, click the “Not Secure” section on the left of the URL bar and - select “Certificate is not valid” to show the certificate. Then click Details - tab and click Export at the bottom. -2. Once you have the certificate, you will need to trust it on your machine. For - MacOS, you can open the Keychain Access app. Select System under System - Keychains. Then go to File > Import Items... and import the certificate you - downloaded in the previous step. -3. Once imported, select the certificate and right-click to select "Get Info". - Expand the Trust section, and change the value of "When using this - certificate" to "Always Trust". -4. You should now refresh the browser and see that you can proceed to open the - application. For Chrome, the application will remain “Not Secure” due to the - fact that this is a self-signed certificate. If you have trouble accessing - Graph Explorer after completing the previous step and reloading the browser, - consider running a docker restart command and refreshing the browser again. - - -> [!TIP] -> -> To get rid of the “Not Secure” warning, see -[Using self-signed certificates on Chrome](./additionaldocs/development.md#using-self-signed-certificates-on-chrome). - -## Permissions - -Graph Explorer does not provide any mechanisms for controlling user permissions. -If you are using Graph Explorer with AWS, Neptune permissions can be controlled -through IAM roles. - -For information about what permissions Graph Explorer requires check out the -documentation on -[SageMaker configuration](./additionaldocs/sagemaker/README.md#minimum-database-permissions). - - -> [!CAUTION] -> -> By default, a Neptune Notebook will have full read & write access to Neptune data. - -## Authentication - -Authentication for Amazon Neptune connections is enabled using the -[SigV4 signing protocol](https://docs.aws.amazon.com/general/latest/gr/signature-version-4.html). - -To use AWS IAM authentication, you must run requests through a proxy endpoint, -such as an EC2 instance, where credentials are resolved and where requests are -signed. - -To set up a connection in Graph Explorer UI with AWS IAM auth enabled on -Neptune, check Using Proxy-Server, then check AWS IAM Auth Enabled and type in -the AWS Region where the Neptune cluster is hosted (e.g., us-east-1). - -For further information on how AWS credentials are resolved in Graph Explorer, -refer to this -[documentation](https://docs.aws.amazon.com/AWSJavaScriptSDK/latest/AWS/CredentialProviderChain.html). - -## Health Check Status - -The `graph-explorer-proxy-server` provides a `/status` endpoint for monitoring -its health and readiness. This endpoint is crucial for ensuring reliable service -operation and can be utilized in various deployment scenarios. - -**Key Features:** - -- **Health Check:** The `/status` endpoint serves as a basic health check, - confirming that the Express server is running and responding. This is - essential for load balancers (like AWS ALB) to determine if the server is - operational and should receive traffic. -- **Readiness Probe:** It also functions as a readiness probe in container - orchestration systems (like Kubernetes). This allows the orchestrator to know - when the server is ready to accept requests, preventing traffic from being - routed to instances that are still starting up or experiencing issues. -- **Expected Response:** A successful health check or readiness probe will - result in an HTTP `200 OK` response with the body containing `OK`. - -## Logging - -Logs are, by default, sent to the console and will be visible as output to the -docker logs. If you want to access the full set of logs, you can run -`docker logs {container name or id}`. - -The log level will be set via the `LOG_LEVEL` env variable at -`/packages/graph-explorer/.env` where the possible options, from highest to -lowest, are `error`, `warn`, `info`, `debug`, and `trace` such that `error` is -the highest level and will only include logs labeled as errors and `trace` the -lowest and will include any type of log. - -By default, the log level is set to `info` and the only type of logs generated -are those of `error`, `info`, or `debug`. If you need more detailed logs, you -can change the log level from `info` in the default .env file to `debug` and the -logs will begin printing the error's stack trace. - -Within node-server.js, you'll notice three things. - -1. A `proxyLogger` object - This is responsible for actually recording the logs. -2. An `errorHandler` - This automatically sends errors to the `proxyLogger` and - can log extra information by adding wanted text to the error object at a key - called `extraInfo`. -3. An endpoint called `/logger` - This is how you would log things from the - browser. It needs a log level and message header passed and you can then - expect to see the message logged at the provided log level. +For development guidance, see [Development](./docs/development/). ## Contributing Guidelines diff --git a/additionaldocs/README.md b/additionaldocs/README.md deleted file mode 100644 index 40092c509..000000000 --- a/additionaldocs/README.md +++ /dev/null @@ -1,6 +0,0 @@ -## Table of Contents - -- [Connections](./connections.md) -- [Development](./development.md) -- [SageMaker](./sagemaker) -- [ECS](./ecs) diff --git a/additionaldocs/features/README.md b/additionaldocs/features/README.md deleted file mode 100644 index fe28056af..000000000 --- a/additionaldocs/features/README.md +++ /dev/null @@ -1,247 +0,0 @@ -# Features - -If you are interested in where Graph Explorer is headed in the future then check -out our [roadmap](../../ROADMAP.md) and participate in the discussions. - -## Settings UI - -### General Settings - -- **Default Neighbor Expansion Limit:** This setting will allow you to enable or - disable the default limit applied during neighbor expansion. This applies to - both double click expansion and the expand sidebar. This setting can be - overridden by a similar setting on the connection itself. -- **Save Configuration:** This action will export all the configuration data - within the Graph Explorer local database. This will not store any data from - the connected graph databases. However, the export may contain the shape of - the schema for your databases and the connection URL. -- **Load Configuration:** This action will replace all the Graph Explorer - configuration data you currently have with the data in the provided - configuration file. This is a destructive act and can not be undone. It is - **strongly** suggested that you perform a **Save Configuration** action before - performing a **Load Configuration** action to preserve any existing - configuration data. - -### About - -In the _About_ page you can see the version number and submit any feedback. - -## Connections UI - -You can create and manage connections to graph databases using this feature. -Connections is accessible as the first screen after deploying the application, -when you click `Open Connections` on the top-right. Click `+` on the top-right -to add a new connection. You can also edit and delete connections. - -- **Add a new connection:** - - **Name:** Enter a name for your connection (e.g., `MyNeptuneCluster`). - - **Graph Type:** Choose a graph data model that corresponds to your graph - database. - - **Public or proxy endpoint:** Provide the publicly accessible endpoint URL - for a graph database, e.g., Gremlin Server. If connecting to Amazon Neptune, - then provide a proxy endpoint URL that is accessible from outside the VPC, - e.g., EC2. - - **Note:** For connecting to Amazon Neptune, ensure that the graph - connection URL is in the format `https://[NEPTUNE_ENDPOINT]:8182`, and - that the proxy endpoint URL is either `https://[EC2_PUBLIC_HOSTNAME]:443` - or `http://[EC2_PUBLIC_HOSTNAME]:80`, depending on the protocol used. - Ensure that you don't end either of the URLs with `/`. - - **Using proxy server:** Check this box if using a proxy endpoint. - - **Graph connection URL:** Provide the endpoint for the graph database - - **AWS IAM Auth Enabled:** Check this box if connecting to Amazon Neptune - using IAM Auth and SigV4 signed requests - - **Service Type:** Choose the service type - - **AWS Region:** Specify the AWS region where the Neptune cluster is hosted - (e.g., us-east-1) - - **Fetch Timeout:** Specify the timeout for the fetch request - - **Neighbor Expansion Limit:** Specify the default limit for neighbor - expansion. This will override the app setting for neighbor expansion. - -- **Available Connections:** Once a connection is created, this section will - appear as a left-hand pane. When you create more than one connection to a - graph database, you can only connect to and visualize from one graph database - endpoint at a time. To select the active database, toggle the “Active” switch. - -- **Connection Details:** Once a connection is created, this section will appear - as a right-hand information pane for a selected connection. It shows details - such as the connection name, graph data model type, endpoint and a summary of - the graph data, such as the count of nodes, edges, and a list of node types. -- **Last Synchronization:** When a connection is created, Graph Explorer will - perform a scan of the graph to provide summary data. To re-synchronize after - data has changed on your graph, select a connection, and then click the - “refresh” button next to “Last Synchronization” text. -- **Data Explorer UI:** Under a listed node type, you can click on the ‘>’ arrow - to get to the “Data Explorer” view. This allows you to see a sample list of - nodes under this type and choose one or more nodes to “Send to Explorer” for - getting started quickly if you are new to the data. - -## Graph Explorer UI - -You can search, browse, expand, customize views of your graph data using Graph -Explorer, which is the main UI of this application. Once you create a -connection, you can click “Open Graph Explorer” on the top-right to navigate -here. There are several key features on this UI: - -### Top Bar UI - -- **Toggles:** You can toggle to show/hide the Graph View and/or Table View for - screen real-estate management. -- **Open Connections:** This takes the user back to Connections UI. - -### Graph View UI - -The graph visualization canvas that you can interact with. Double-click to -expand the first-order neighbors of a node. - -- **Layout drop-down & reset:** You can display graph data using standard graph - layouts in the Graph View. You can use the circular arrow to reset the physics - of a layout. -- **Screenshot:** Download a picture of the current window in Graph View. -- **Save Graph:** Save the current rendered graph as a JSON file that can be - shared with others having the same connection or reloaded at a later time. -- **Load Graph:** Load a previously saved graph from a JSON file. -- **Zoom In/Out & Clear:** To help users quickly zoom in/out or clear the whole - canvas in the Graph View. -- **Legend (i):** This displays an informational list of icons, colors, and - display names available. - -### Sidebar Panel UI - -The panel on the right of the graph provides various actions, configuration, and -details about the open graph. - -- [**Search panel**](#search-panel) allows you to search for specific nodes by - filtering on node types & attributes or executing a database query then adding - nodes & edges to the graph panel. -- [**Details panel**](#details-panel) shows details about a selected node/edge - such as properties etc. -- [**Entities filter panel**](#entities-filter-panel) is used to control the - display of nodes and edges that are already expanded in the Graph View; click - to hide or show nodes/edges. -- [**Expand panel**](#expand-panel) provides controls and filters to help focus - large neighbor expansions. -- [**Node styling panel**](#node-styling-panel) of node display options (e.g., - color, icon, the property to use for the displayed name). -- [**Edge styling panel**](#edge-styling-panel) of edge display options (e.g., - color, icon, the property to use for the displayed name). -- [**Namespaces panel (RDF only)**](#namespace-panel) allows you to shorten the - display of Resource URIs within the app based on auto-generated prefixes, - commonly-used prefix libraries, or custom prefixes set by the user. Order of - priority is set to Custom > Common > Auto-generated. - -#### Search Panel - -Graph Explorer Search UI provides two powerful ways to search and interact with -your graph database: - -##### Filter Search - -- Enables faceted filtering of nodes based on: - - Node labels (or rdf:type for RDF databases) - - Node attribute values -- Supports partial text matching -- Search results can be added to the graph individually or all at once -- Supports cancellation of long-running queries - -##### Query Search - -- Available for Gremlin and openCypher connections -- Allows execution of any valid database query, including mutations -- When adding an edge, its connected nodes are automatically included -- Displays scalar values in results (though these cannot be added to the graph) -- Paginates large result sets -- Results can be added to the graph individually or all at once -- Supports cancellation of long-running queries - -#### Details Panel - -- Displays attributes and values for node or edge selections -- Displays neighbor counts by label for node selections -- Displays relationship information for edge selections - -#### Expand Panel - -Provides fine grained control over neighbor expansions - -- Filter by node label (or rdf:type for RDF databases) -- Filter by attribute value -- Limit results to a maximum size - -#### Entities Filter Panel - -Provides the ability to filter nodes or edges from the visualization by label -(or rdf:type for RDF databases) - -#### Node Styling Panel - -Each node type can be customized in a variety of ways. - -- **Display label** allows you to change how the node label (or rdf:type) is - represented -- **Display name attribute** allows you to choose the attribute on the node that - is used to uniquely label the node in the graph visualization and search -- **Display description attribute** allows you to choose the attribute on the - node that is used to describe the node in search -- **Custom symbol** can be uploaded in the form of an SVG icon -- **Colors and borders** can be customized to visually distinguish from other - node types - -#### Edge Styling Panel - -Each edge type can be customized in a variety of ways. - -- **Display label** allows you to change how the edge label (or predicate in RDF - databases) is represented -- **Display name attribute** allows you to choose the attribute on the edge that - is used to uniquely label the edge in the graph visualization and search -- **Arrow symbol** can be chosen for both source and target variations -- **Colors and borders** can be customized for the edge label and the line -- **Line style** can be solid, dotted, or dashed - -#### Namespace Panel - -- Only visible in RDF connections -- Displays any automatically generated namespace prefixes for the connection -- Displays all the common prefixes that are built-in -- Allows creation and management of any custom namespace prefixes - -### Table View UI - -This collapsible view shows a row-column display of the data in the Graph View. -You can use filters in the Table to show/hide elements in the Graph View, and -you can export the table view into a CSV or JSON file. - -The following columns are available for filtering on property graphs (RDF graphs -in parentheses): - -- Node ID (Resource URI) -- Node Type (Class) -- Edge Type (Predicate) -- Source ID (Source URI) -- Source Type (Source Class) -- Target ID (Target URI) -- Target Type (Target Class) -- Display Name - Set in the Node/Edge Styling panes -- Display Description - Set in the Node/Edge Styling panes -- Total Neighbors - Enter an integer to be used as the >= limit - -#### Additional Table View UI Features - -- **Visibility** - manually show or hide nodes or edges -- **All Nodes / All Edges (or All Resources / All Predicates) dropdown** - - allows you to display a list of either nodes or edges and control - display/filter on them -- **Download** - You can download the current Table View as a CSV or JSON file - with additional customization options -- **Default columns** - You can set which columns you want to display -- Paging of rows - -## Data Explorer UI - -You can use the Data Explorer UI to view the data for the selected node type. -You can open the Data Explorer by clicking the node type row in the connection -details pane. - -- View tabular data for the selected node type -- Set the node type display name and description attributes -- Send a specific node to the graph view diff --git a/additionaldocs/getting-started/README.md b/additionaldocs/getting-started/README.md deleted file mode 100644 index 6b190767f..000000000 --- a/additionaldocs/getting-started/README.md +++ /dev/null @@ -1,208 +0,0 @@ -# Getting Started - -This project contains the code needed to create a Docker image of the Graph -Explorer. The image will create the Graph Explorer application and proxy server -that will be served over the standard HTTP or HTTPS ports (HTTPS by default). - -The proxy server will be created automatically, but will only be necessary if -you are connecting to Neptune. Gremlin-Server and BlazeGraph can be connected to -directly. Additionally, the image will create a self-signed certificate that can -be optionally used. - -## Examples - -### Local Docker Setup - -The quickest way to get started with Graph Explorer is to use the official -Docker image. You can find the latest version of the image on -[Amazon's ECR Public Registry](https://gallery.ecr.aws/neptune/graph-explorer). - - -> [!NOTE] -> -> Make sure to use the version of the image that does not include `sagemaker` in the tag. - -#### Prerequisites - -- [Docker](https://docs.docker.com/get-docker/) installed on your machine -- [AWS CLI](https://docs.aws.amazon.com/cli/latest/userguide/install-cliv2.html) - installed on your machine - -#### Steps - -1. Authenticate with the Amazon ECR Public Registry. - [More information](https://docs.aws.amazon.com/AmazonECR/latest/public/public-registries.html#public-registry-auth) - - ``` - aws ecr-public get-login-password --region us-east-1 | docker login --username AWS --password-stdin public.ecr.aws - ``` - -2. Pull down the docker image - - ``` - docker pull public.ecr.aws/neptune/graph-explorer - ``` - -3. Create and run a docker container using the image - - ``` - docker run -p 80:80 -p 443:443 \ - --env HOST=localhost \ - --name graph-explorer \ - public.ecr.aws/neptune/graph-explorer - ``` - - The `HOST` environment variable is used for SSL certificates generation since - HTTPS is the default. If you are hosting this on a public domain, you should - replace `HOST=localhost` with your domain name. - -4. Open a browser and type in the URL of the Graph Explorer server instance - - ``` - https://localhost/explorer - ``` - -5. You will receive a warning as the SSL certificate used is self-signed. Since - the application is set to use HTTPS by default and contains a self-signed - certificate, you will need to add the Graph Explorer certificates to the - trusted certificates directory and manually trust them. See the - [HTTPS Connections](../troubleshooting.md#https-connections) section. -6. After completing the trusted certification step and refreshing the browser, - you should now see the Connections UI. See below description on Connections - UI to configure your first connection to Amazon Neptune. - -#### Gremlin Server Database - -Gremlin Server is an easy way to get started with graph databases. This example -will configure a simple Gremlin Server instance to be used with Graph Explorer. -It comes with a very small graph dataset. - -1. Pull the latest Gremlin Server image from Docker Hub. - ``` - docker pull tinkerpop/gremlin-server:latest - ``` -2. Create and run the Gremlin Server container using the HTTP REST modern - configuration. - ``` - docker run -p 8182:8182 \ - --name gremlin-server \ - tinkerpop/gremlin-server:latest \ - conf/gremlin-server-rest-modern.yaml - ``` -3. Open Graph Explorer and add a new connection - - Name: `Gremlin Server` - - Graph Type: `Gremlin` - - Public or Proxy Endpoint: `https://localhost` - - Using Proxy Server: `true` - - Graph Connection URL: `http://localhost:8182` - -## Amazon EC2 Setup - -The following instructions detail how to deploy graph-explorer onto an Amazon -EC2 instance and use it as a proxy server with SSH tunneling to connect to -Amazon Neptune. - - -> [!NOTE] -> -> This documentation is not an official recommendation on -network setups as there are many ways to connect to Amazon Neptune from outside -of the VPC, such as setting up a load balancer or VPC peering. - -### Prerequisites - -- Provision an Amazon EC2 instance that will be used to host the application and - connect to Neptune as a proxy server. For more details, see instructions - [here](https://github.com/aws/graph-notebook/tree/main/additional-databases/neptune). -- Ensure the Amazon EC2 instance can send and receive on ports `22` (SSH), - `8182` (Neptune), and `443` or `80` depending on protocol used - (graph-explorer). - -### Steps - -These steps describe how to install Graph Explorer on your Amazon EC2 instance. - -1. Open an SSH client and connect to the EC2 instance. -2. Download and install the necessary command line tools such as - [Git](https://git-scm.com/downloads) and - [Docker](https://docs.docker.com/get-docker/). -3. Clone the repository - ``` - git clone https://github.com/aws/graph-explorer.git - ``` -4. Navigate to the repository - ``` - cd graph-explorer - ``` -5. Build the image - ``` - docker build -t graph-explorer . - ``` - - -> [!TIP] -> -> If you receive an error relating to the docker service not running, run -> `service docker start`. - -4. Run the container substituting the `{hostname-or-ip-address}` with the - hostname or IP address of the EC2 instance. - ``` - docker run -p 80:80 -p 443:443 \ - --env HOST={hostname-or-ip-address} \ - graph-explorer - ``` -5. Navigate to the public URL of your EC2 instance accessing the `/explorer` - endpoint. You will receive a warning as the SSL certificate used is - self-signed. The URL will look like this: - ``` - https://ec2-1-2-3-4.us-east-1.compute.amazonaws.com/explorer - ``` -6. Since the application is set to use HTTPS by default and contains a - self-signed certificate, you will need to add the Graph Explorer certificates - to the trusted certificates directory and manually trust them. See - [HTTPS Connections](#https-connections) section. -7. After completing the trusted certification step and refreshing the browser, - you should now see the Connections UI. - -## Local Development Setup - -You can build the Docker image locally by following the steps below. - -### Prerequisites - -- [Docker](https://docs.docker.com/get-docker/) installed on your machine -- [Git](https://git-scm.com/downloads) installed on your machine - -### Steps - -1. Clone the repository - ``` - git clone https://github.com/aws/graph-explorer.git - ``` -2. Navigate to the repository - ``` - cd graph-explorer - ``` -3. Build the image - ``` - docker build -t graph-explorer . - ``` -4. Run the container (HTTPS disabled) - ``` - docker run -p 80:80 \ - --name graph-explorer \ - --env PROXY_SERVER_HTTPS_CONNECTION=false \ - --env GRAPH_EXP_HTTPS_CONNECTION=false \ - graph-explorer - ``` -5. Connect to the Graph Explorer UI - ``` - http://localhost/explorer - ``` - -## Troubleshooting - -If the instructions above do not work for you, please see the -[Troubleshooting](../troubleshooting.md) page for more information. It contains -workarounds for common issues and information on how to diagnose other issues. diff --git a/additionaldocs/troubleshooting.md b/additionaldocs/troubleshooting.md deleted file mode 100644 index fdd5f15f3..000000000 --- a/additionaldocs/troubleshooting.md +++ /dev/null @@ -1,231 +0,0 @@ -# Troubleshooting - -This page contains workarounds for common issues and information on how to -diagnose other issues. - -- [Docker Container Issues](#docker-container-issues) -- [Schema Sync Fails](#schema-sync-fails) -- [Backup Graph Explorer Data](#backup-graph-explorer-data) -- [Gathering SageMaker Logs](#gathering-sagemaker-logs) - -## Docker Container Issues - -1. If the container does not start, or immediately stops, use - `docker logs graph-explorer` to check the container console logs for any - related error messages that might provide guidance on why graph-explorer did - not start. -2. If you are having issues connecting graph-explorer to your graph database, - use your browser's Developer Tools feature to monitor both the browser - console and network calls to determine if here are any errors related to - connectivity. - -### Ports in Use - -If the default ports of 80 and 443 are already in use, you can use the `-p` flag -to change the ports to something else. The host machine ports are the first of -the two numbers. You can change those to whatever you want. - -For example, if you want to use port 8080 and 4431, you can use the following -command: - -``` -docker run -p 8080:80 -p 4431:443 \ - --name graph-explorer \ - --env HOST=localhost \ - public.ecr.aws/neptune/graph-explorer -``` - -Which will result in the following URLs: - -- HTTPS: `https://localhost:4431/explorer` -- HTTP: `http://localhost:8080/explorer` - -### HTTP Only - -If you do not want to use SSL and HTTPS, you can disable it by setting the -following -[environment variables](/additionaldocs/development.md#environment-variables): - -``` -PROXY_SERVER_HTTPS_CONNECTION=false -GRAPH_EXP_HTTPS_CONNECTION=false -``` - -These can be passed when creating the Docker container like so: - -``` -docker run -p 80:80 \ - --name graph-explorer \ - --env PROXY_SERVER_HTTPS_CONNECTION=false \ - --env GRAPH_EXP_HTTPS_CONNECTION=false \ - public.ecr.aws/neptune/graph-explorer -``` - -### HTTPS Connections - -If either of the Graph Explorer or the proxy-server are served over an HTTPS -connection (which it is by default), you will have to bypass the warning message -from the browser due to the included certificate being a self-signed -certificate. - -You can bypass by manually ignoring them from the browser or downloading the -correct certificate and configuring them to be trusted. Alternatively, you can -provide your own certificate. - -The following instructions can be used as an example to bypass the warnings for -Chrome, but note that different browsers and operating systems will have -slightly different steps. - -1. Download the certificate directly from the browser. For example, if using - Google Chrome, click the “Not Secure” section on the left of the URL bar and - select “Certificate is not valid” to show the certificate. Then click Details - tab and click Export at the bottom. -2. Once you have the certificate, you will need to trust it on your machine. For - MacOS, you can open the Keychain Access app. Select System under System - Keychains. Then go to File > Import Items... and import the certificate you - downloaded in the previous step. -3. Once imported, select the certificate and right-click to select "Get Info". - Expand the Trust section, and change the value of "When using this - certificate" to "Always Trust". -4. You should now refresh the browser and see that you can proceed to open the - application. For Chrome, the application will remain “Not Secure” due to the - fact that this is a self-signed certificate. If you have trouble accessing - Graph Explorer after completing the previous step and reloading the browser, - consider running a docker restart command and refreshing the browser again. - - -> [!TIP] -> To get rid of the “Not Secure” warning, see -[Using self-signed certificates on Chrome](../development.md#using-self-signed-certificates-on-chrome). - -## Schema Sync Fails - -This can happen for many reasons. Below are a few of the common ones. - -### Mismatched Proxy Server URL - -When running a schema sync, a common cause of failures is a mismatch between the -proxy server URL in your connection settings and the URL currently loaded in -your browser. This mismatch can trigger a "Connection Error" message in Graph -Explorer. - -Modern browsers enforce a security policy called -[Same-Origin Policy](https://developer.mozilla.org/en-US/docs/Web/Security/Same-origin_policy), -which requires API requests to be sent to the same domain as the page you're -viewing. If your connection settings specify a different domain than your -current browser URL, the request will be blocked. - -To resolve this issue: - -1. Check your connection settings -2. Ensure the proxy server URL matches the domain portion of your current - browser URL -3. Update the connection if necessary - -For example: - -``` -# if the current browser page URL is -https://graph-explorer.mydomain.com/explorer/#/connections - -# the connection setting's proxy server URL should be -https://graph-explorer.mydomain.com -``` - -### Timeout - -There are multiple sources of timeouts. - -- Database server -- Networking layer (load balancer, proxy, etc) -- Browser -- Graph Explorer connection configuration - -The error you receive in Graph Explorer can interpret Neptune timeout errors and -timeouts from Graph Explorer's configuration. - -### Out of Memory - -This can happen when your database is very large. Graph Explorer does its best -to support larger databases and is always improving. Please -[file an issue](https://github.com/aws/graph-explorer/issues/new/choose) if you -encounter this situation. - -### Proxy Server Cannot Be Reached - -Communication between the client and proxy server can be configured in different -ways. When Graph Explorer proxy server starts up it will print out its best -approximation of the correct public proxy server address. - -This can manifest as different types of errors depending on the root cause. You -may receive 404 not found responses or get connection refused errors. - -- The proxy server can be hosting HTTP or HTTPS -- The port of the proxy server could be the default (i.e. 80 or 443 with SSL) or - a specific port provided through environment values -- The proxy server paths are not exposed by the networking layer (load balancer, - proxy, firewall, etc) - - The client is hosted at `/explorer` by default, which is configurable - - Queries are handled via `/gremlin`, `/opencypher`, `/sparql` - - Summary APIs are handled via `/summary`, `/pg/statistics/summary`, - `/rdf/statistics/summary` - - Logging is handled by `/logger` - - Default connection is handled by `/defaultConnection` - -> [!IMPORTANT] -> The paths listed here could always change in the future. If they do change, we -> will note that in the release notes. - -## Backup Graph Explorer Data - -Inside of Graph Explorer there is an option to export all the configuration data -that Graph Explorer uses. This data is local to the user’s browser and does not -exist on the server. - -To gather the config data: - -1. Launch Graph Explorer -2. Navigate to the connections screen -3. Press the “Settings” button in the navigation bar -4. Select the “General” page within settings -5. Press the “Save Configuration” button -6. Choose where to save the exported file - -This backup can be restored using the “Load Configuration” button in the same -settings page. - -## Gathering SageMaker Logs - -The Graph Explorer proxy server outputs log statements to standard out. By -default, these logs will be forwarded to CloudWatch if the Notebook has the -proper permissions. - -To gather these logs: - -1. Open the AWS Console -2. Navigate to the Neptune page -3. Select “Notebook” from the sidebar -4. Find the Notebook hosting Graph Explorer -5. Open the details screen for that Notebook -6. In the “Actions” menu, choose “View Details in SageMaker” -7. Press the “View Logs” link in the SageMaker details screen under the field - titled “Lifecycle configuration” -8. Scroll down to the “Log Streams” panel in the CloudWatch details where you - should find multiple log streams -9. For each log stream related to Graph Explorer (LifecycleConfigOnStart.log, - graph-explorer.log) - 1. Open the log stream - 2. In the “Actions” menu, choose “Download search results (CSV)” - -### Graph Explorer Log Stream Does Not Exist - -New Neptune Notebooks automatically apply the correct IAM permissions to write -to CloudWatch. If your Notebook does not automatically create a -graph-explorer.log in the CloudWatch Log Streams, then it is possible that the -Neptune Notebook was created before those IAM permissions were added. You’ll -need to add those permissions manually. - -Below are examples of which IAM permissions you need for Graph Explorer. - -- [IAM permissions for Neptune DB](https://raw.githubusercontent.com/aws/graph-explorer/main/additionaldocs/sagemaker/graph-explorer-neptune-db-policy.json) -- [IAM permissions for Neptune Analytics](https://raw.githubusercontent.com/aws/graph-explorer/main/additionaldocs/sagemaker/graph-explorer-neptune-analytics-policy.json) diff --git a/docs/README.md b/docs/README.md new file mode 100644 index 000000000..a7449aaef --- /dev/null +++ b/docs/README.md @@ -0,0 +1,8 @@ +## Table of Contents + +- [Getting Started](./getting-started) +- [User Guide](./user-guide) +- [Deployment](./deployment) +- [Administration](./administration) +- [Development](./development) +- [Troubleshooting](./troubleshooting/) diff --git a/docs/administration/README.md b/docs/administration/README.md new file mode 100644 index 000000000..eba4054e5 --- /dev/null +++ b/docs/administration/README.md @@ -0,0 +1,11 @@ +# Administration + +This section contains guides for administrators and operators who need to +manage, monitor, and maintain Graph Explorer deployments. + +## Files in this section + +1. [Monitoring](monitoring.md) - Health checks, logging, and monitoring + capabilities +2. [Backup and Restore](backup-restore.md) - Backing up and restoring Graph + Explorer configuration diff --git a/docs/administration/backup-restore.md b/docs/administration/backup-restore.md new file mode 100644 index 000000000..aed8bfd30 --- /dev/null +++ b/docs/administration/backup-restore.md @@ -0,0 +1,21 @@ +# Backup and Restore + +This guide covers how to backup and restore Graph Explorer configuration data. + +## Backup Graph Explorer Data + +Inside of Graph Explorer there is an option to export all the configuration data +that Graph Explorer uses. This data is local to the user's browser and does not +exist on the server. + +To gather the config data: + +1. Launch Graph Explorer +2. Navigate to the connections screen +3. Press the "Settings" button in the navigation bar +4. Select the "General" page within settings +5. Press the "Save Configuration" button +6. Choose where to save the exported file + +This backup can be restored using the "Load Configuration" button in the same +settings page. diff --git a/docs/administration/monitoring.md b/docs/administration/monitoring.md new file mode 100644 index 000000000..b84d68286 --- /dev/null +++ b/docs/administration/monitoring.md @@ -0,0 +1,77 @@ +# Monitoring + +This guide covers monitoring capabilities, health checks, and logging +configuration for Graph Explorer across different deployment environments. + +## Health Check Status + +The `graph-explorer-proxy-server` provides a `/status` endpoint for monitoring +its health and readiness. This endpoint is crucial for ensuring reliable service +operation and can be utilized in various deployment scenarios. + +**Key Features:** + +- **Health Check:** The `/status` endpoint serves as a basic health check, + confirming that the Express server is running and responding. This is + essential for load balancers (like AWS ALB) to determine if the server is + operational and should receive traffic. +- **Readiness Probe:** It also functions as a readiness probe in container + orchestration systems (like Kubernetes). This allows the orchestrator to know + when the server is ready to accept requests, preventing traffic from being + routed to instances that are still starting up or experiencing issues. +- **Expected Response:** A successful health check or readiness probe will + result in an HTTP `200 OK` response with the body containing `OK`. + +## Logging + +Logs are, by default, sent to the console and will be visible as output to the +docker logs. If you want to access the full set of logs, you can run +`docker logs {container name or id}`. + +The log level will be set via the `LOG_LEVEL` env variable at +`/packages/graph-explorer/.env` where the possible options, from highest to +lowest, are `error`, `warn`, `info`, `debug`, and `trace` such that `error` is +the highest level and will only include logs labeled as errors and `trace` the +lowest and will include any type of log. + +By default, the log level is set to `info` and the only type of logs generated +are those of `error`, `info`, or `debug`. If you need more detailed logs, you +can change the log level from `info` in the default .env file to `debug` and the +logs will begin printing the error's stack trace. + +Within node-server.js, you'll notice three things. + +1. A `proxyLogger` object - This is responsible for actually recording the logs. +2. An `errorHandler` - This automatically sends errors to the `proxyLogger` and + can log extra information by adding wanted text to the error object at a key + called `extraInfo`. +3. An endpoint called `/logger` - This is how you would log things from the + browser. It needs a log level and message header passed and you can then + expect to see the message logged at the provided log level. + +## Gathering SageMaker Logs + +The Graph Explorer proxy server outputs log statements to standard out. By +default, these logs will be forwarded to CloudWatch if the Notebook has the +proper permissions. + +To gather these logs: + +1. Open the AWS Console +2. Navigate to the Neptune page +3. Select "Notebook" from the sidebar +4. Find the Notebook hosting Graph Explorer +5. Open the details screen for that Notebook +6. In the "Actions" menu, choose "View Details in SageMaker" +7. Press the "View Logs" link in the SageMaker details screen under the field + titled "Lifecycle configuration" +8. Scroll down to the "Log Streams" panel in the CloudWatch details where you + should find multiple log streams +9. For each log stream related to Graph Explorer (LifecycleConfigOnStart.log, + graph-explorer.log) + 1. Open the log stream + 2. In the "Actions" menu, choose "Download search results (CSV)" + +If you encounter issues with missing log streams, see the +[SageMaker troubleshooting](../deployment/aws-sagemaker.md#troubleshooting) +section. diff --git a/docs/deployment/README.md b/docs/deployment/README.md new file mode 100644 index 000000000..659aa4d45 --- /dev/null +++ b/docs/deployment/README.md @@ -0,0 +1,20 @@ +# Deployment + +This section contains guides for deploying Graph Explorer in production +environments. + +## Files in this section + +1. [Database Setup](database-setup.md) - Configure different graph database + engines +2. [Default Connection](default-connection.md) - Configure automatic default + connections +3. [HTTPS Setup](https-setup.md) - HTTPS configuration and certificate + management +4. [Docker](docker.md) - Deploy Graph Explorer locally using Docker +5. [AWS EC2](aws-ec2.md) - Deploy Graph Explorer on Amazon EC2 instances +6. [AWS ECS Fargate](aws-ecs-fargate.md) - Deploy Graph Explorer on AWS ECS + using Fargate +7. [AWS SageMaker](aws-sagemaker.md) - Deploy Graph Explorer on Amazon SageMaker + Notebooks +8. [Configuration](configuration.md) - Environment variables and settings diff --git a/docs/deployment/aws-ec2.md b/docs/deployment/aws-ec2.md new file mode 100644 index 000000000..a3f02de25 --- /dev/null +++ b/docs/deployment/aws-ec2.md @@ -0,0 +1,68 @@ +# Deploying Graph Explorer on Amazon EC2 + +The following instructions detail how to deploy Graph Explorer onto an Amazon +EC2 instance and use it as a proxy server with SSH tunneling to connect to +Amazon Neptune. + + +> [!NOTE] +> +> This documentation is not an official recommendation on +network setups as there are many ways to connect to Amazon Neptune from outside +of the VPC, such as setting up a load balancer or VPC peering. + +## Prerequisites + +- Provision an Amazon EC2 instance that will be used to host the application and + connect to Neptune as a proxy server. For more details, see instructions + [here](https://github.com/aws/graph-notebook/tree/main/additional-databases/neptune). +- Ensure the Amazon EC2 instance can send and receive on ports `22` (SSH), + `8182` (Neptune), and `443` or `80` depending on protocol used + (graph-explorer). + +## Steps + +These steps describe how to install Graph Explorer on your Amazon EC2 instance. + +1. Open an SSH client and connect to the EC2 instance. +2. Download and install the necessary command line tools such as + [Git](https://git-scm.com/downloads) and + [Docker](https://docs.docker.com/get-docker/). +3. Clone the repository + ``` + git clone https://github.com/aws/graph-explorer.git + ``` +4. Navigate to the repository + ``` + cd graph-explorer + ``` +5. Build the image + ``` + docker build -t graph-explorer . + ``` + + +> [!TIP] +> +> If you receive an error relating to the docker service not running, run +> `service docker start`. + +4. Run the container substituting the `{hostname-or-ip-address}` with the + hostname or IP address of the EC2 instance. + ``` + docker run -p 80:80 -p 443:443 \ + --env HOST={hostname-or-ip-address} \ + graph-explorer + ``` +5. Navigate to the public URL of your EC2 instance accessing the `/explorer` + endpoint. You will receive a warning as the SSL certificate used is + self-signed. The URL will look like this: + ``` + https://ec2-1-2-3-4.us-east-1.compute.amazonaws.com/explorer + ``` +6. Since the application is set to use HTTPS by default and contains a + self-signed certificate, you will need to add the Graph Explorer certificates + to the trusted certificates directory and manually trust them. See the + [HTTPS Connections](https-setup.md#https-connections) section. +7. After completing the trusted certification step and refreshing the browser, + you should now see the Connections UI. diff --git a/additionaldocs/ecs/ECS_FARGATE_DEPLOYMENT.md b/docs/deployment/aws-ecs-fargate.md similarity index 99% rename from additionaldocs/ecs/ECS_FARGATE_DEPLOYMENT.md rename to docs/deployment/aws-ecs-fargate.md index f9ffe4a50..9bab00f5b 100644 --- a/additionaldocs/ecs/ECS_FARGATE_DEPLOYMENT.md +++ b/docs/deployment/aws-ecs-fargate.md @@ -195,8 +195,7 @@ the specified domain names. - `taskRoleArn` and `executionRoleArn`: The ARN of the IAM role created in step "Create a new IAM role and permission policies". - `environment` variables section (see - [Default Connections](https://github.com/aws/graph-explorer#providing-a-default-connection) - for more details): + [Default Connections](default-connection.md) for more details): - `AWS_REGION`: The AWS region in which your Neptune cluster is located. - `GRAPH_TYPE`: The query language for your initial connection. - `IAM`: Set this to `true` to use SigV4 signed requests, if your Neptune diff --git a/additionaldocs/sagemaker/README.md b/docs/deployment/aws-sagemaker.md similarity index 69% rename from additionaldocs/sagemaker/README.md rename to docs/deployment/aws-sagemaker.md index 00d14e1c8..969a3458b 100644 --- a/additionaldocs/sagemaker/README.md +++ b/docs/deployment/aws-sagemaker.md @@ -6,12 +6,12 @@ how to create one, see the [documentation](https://docs.aws.amazon.com/sagemaker/latest/dg/notebook-lifecycle-config.html). You can use the provided sample lifecycle configuration, -[`install-graph-explorer-lc.sh`](install-graph-explorer-lc.sh), or create your -own shell script. If using the sample lifecycle, you should also create an IAM -role with a policy containing the permissions described in either -[`graph-explorer-neptune-db-policy.json`](graph-explorer-neptune-db-policy.json) +[`install-graph-explorer-lc.sh`](aws-sagemaker/install-graph-explorer-lc.sh), or +create your own shell script. If using the sample lifecycle, you should also +create an IAM role with a policy containing the permissions described in either +[`graph-explorer-neptune-db-policy.json`](aws-sagemaker/graph-explorer-neptune-db-policy.json) or -[`graph-explorer-neptune-analytics-policy.json`](graph-explorer-neptune-analytics-policy.json), +[`graph-explorer-neptune-analytics-policy.json`](aws-sagemaker/graph-explorer-neptune-analytics-policy.json), depending on the service used. After you have created the lifecycle configuration and IAM role, you can attach @@ -82,3 +82,18 @@ that request. ] } ``` + +## Troubleshooting + +### Graph Explorer Log Stream Does Not Exist + +New Neptune Notebooks automatically apply the correct IAM permissions to write +to CloudWatch. If your Notebook does not automatically create a +graph-explorer.log in the CloudWatch Log Streams, then it is possible that the +Neptune Notebook was created before those IAM permissions were added. You'll +need to add those permissions manually. + +Below are examples of which IAM permissions you need for Graph Explorer. + +- [IAM permissions for Neptune DB](aws-sagemaker/graph-explorer-neptune-db-policy.json) +- [IAM permissions for Neptune Analytics](aws-sagemaker/graph-explorer-neptune-analytics-policy.json) diff --git a/additionaldocs/sagemaker/graph-explorer-neptune-analytics-policy.json b/docs/deployment/aws-sagemaker/graph-explorer-neptune-analytics-policy.json similarity index 100% rename from additionaldocs/sagemaker/graph-explorer-neptune-analytics-policy.json rename to docs/deployment/aws-sagemaker/graph-explorer-neptune-analytics-policy.json diff --git a/additionaldocs/sagemaker/graph-explorer-neptune-db-policy.json b/docs/deployment/aws-sagemaker/graph-explorer-neptune-db-policy.json similarity index 100% rename from additionaldocs/sagemaker/graph-explorer-neptune-db-policy.json rename to docs/deployment/aws-sagemaker/graph-explorer-neptune-db-policy.json diff --git a/additionaldocs/sagemaker/install-graph-explorer-lc.sh b/docs/deployment/aws-sagemaker/install-graph-explorer-lc.sh similarity index 100% rename from additionaldocs/sagemaker/install-graph-explorer-lc.sh rename to docs/deployment/aws-sagemaker/install-graph-explorer-lc.sh diff --git a/docs/deployment/configuration.md b/docs/deployment/configuration.md new file mode 100644 index 000000000..340019252 --- /dev/null +++ b/docs/deployment/configuration.md @@ -0,0 +1,76 @@ +# Configuration + +This guide covers the environment variables and configuration options available +for Graph Explorer deployment. + +## Environment Variables + +### `GRAPH_EXP_ENV_ROOT_FOLDER` + +Base path used to serve the `graph-explorer` front end application. + +Example: `/explorer` + +- Optional +- Default: `/` +- Type: `string` + +### `GRAPH_EXP_CONNECTION_NAME` + +Default connection name. + +- Optional +- Default is empty +- Type: `string` + +### `GRAPH_EXP_CONNECTION_ENGINE` + +The query engine to use for the default connection. + +- Optional +- Default is `gremlin` +- Valid values are `gremlin`, `openCypher`, or `sparql` +- Type: `string` + +### `HOST` + +The public hostname of the server. This is used to generate the SSL certificate +during the Docker build. + +Example: `localhost` + +- Required when using HTTPS connections +- Default is empty +- Type: `string` + +### `GRAPH_EXP_HTTPS_CONNECTION` + +Uses the self-signed certificate to serve Graph Explorer over https if true. + +- Optional +- Default `true` +- Type: `boolean` + +### `PROXY_SERVER_HTTPS_PORT` + +The port to use for the HTTPS server. + +- Optional +- Default `443` +- Type: `number` + +### `PROXY_SERVER_HTTP_PORT` + +The port to use for the HTTP server. + +- Optional +- Default `80` +- Type: `number` + +### `PROXY_SERVER_HTTPS_CONNECTION` + +Uses the self-signed certificate to serve the proxy-server over https if true. + +- Optional +- Default `true` +- Type: `boolean` diff --git a/additionaldocs/connections.md b/docs/deployment/database-setup.md similarity index 68% rename from additionaldocs/connections.md rename to docs/deployment/database-setup.md index 28580fd92..4fe02182d 100644 --- a/additionaldocs/connections.md +++ b/docs/deployment/database-setup.md @@ -1,16 +1,28 @@ -## Connections +# Database Setup -This section contains detailed instructions that help when configuring Graph -Explorer with different graph database engines. +This guide contains detailed instructions for configuring different graph +database engines to work with Graph Explorer. -### Connecting to Neptune +## Overview + +Graph Explorer supports visualizing both **property graphs** and **RDF graphs**. +You can connect to Amazon Neptune or you can also connect to open graph +databases that implement an Apache TinkerPop Gremlin endpoint or the SPARQL 1.1 +protocol, such as Blazegraph. For additional details on connecting to different +graph databases, see [Connections](../user-guide/connections.md). + +After setting up your database, see the +[Connections UI](../user-guide/connections.md) guide to learn how to create and +manage connections within Graph Explorer. + +## Connecting to Neptune - Ensure that Graph Explorer has access to the Neptune instance by being in the same VPC or VPC peering. - If authentication is enabled, read query privileges are needed (See [ReadDataViaQuery managed policy](https://docs.aws.amazon.com/neptune/latest/userguide/iam-data-access-examples.html#iam-auth-data-policy-example-read-query)). -### Connecting to Gremlin-Server +## Connecting to Gremlin-Server If you are using the default Gremlin Server docker image, you can get the server running with the following commands: @@ -22,7 +34,7 @@ docker run -p 8182:8182 \ conf/gremlin-server-rest-modern.yaml ``` -#### Enable REST +### Enable REST Graph Explorer only supports HTTP(S) connections. When connecting to Gremlin-Server, ensure it is configured with a channelizer that supports HTTP(S) @@ -37,20 +49,20 @@ Gremlin-Server, ensure it is configured with a channelizer that supports HTTP(S) > /conf/gremlin-server.yaml > ``` -#### Versions Prior to 3.7 +### Versions Prior to 3.7 If you have a version of Gremlin Server prior to 3.7, you will need to make the following changes: - **Enable property returns** - Remove - “.withStrategies(ReferenceElementStrategy)” from + ".withStrategies(ReferenceElementStrategy)" from `/scripts/generate-modern.groovy` so that properties are returned. - **Enable string IDs** - Change `gremlin.tinkergraph.vertexIdManager` and `gremlin.tinkergraph.edgeIdManager` in `/conf/tinkergraph-empty.properties` to support string ids. You can use `ANY`. - Build and run the Docker container as normal. -### Connecting to BlazeGraph +## Connecting to BlazeGraph - Build and run the Docker container as normal and connect the proxy-server to BlazeGraph and your workbench to the proxy-server. diff --git a/docs/deployment/default-connection.md b/docs/deployment/default-connection.md new file mode 100644 index 000000000..288921771 --- /dev/null +++ b/docs/deployment/default-connection.md @@ -0,0 +1,98 @@ +# Default Connection + +This guide explains how to configure Graph Explorer with a default connection +that loads automatically when the application starts. + +## Providing a Default Connection + +To provide a default connection such that initial loads of Graph Explorer always +result with the same starting connection, modify the `docker run ...` command to +either take in a JSON configuration or runtime environment variables. If you +provide both a JSON configuration and environmental variables, the JSON will be +prioritized. + +### Environment Variables + +These are the valid environment variables used for the default connection, their +defaults, and their descriptions. + +- Required: + - `PUBLIC_OR_PROXY_ENDPOINT` - `None` - See + [Connections UI](../user-guide/connections.md) +- Optional + - `GRAPH_TYPE` - `None` - If not specified, multiple connections will be + created for every available graph type / query language. See + [Connections UI](../user-guide/connections.md) + - `USING_PROXY_SERVER` - `False` - See + [Connections UI](../user-guide/connections.md) + - `IAM` - `False` - See [Connections UI](../user-guide/connections.md) + - `GRAPH_EXP_HTTPS_CONNECTION` - `True` - Controls whether Graph Explorer uses + SSL or not + - `PROXY_SERVER_HTTPS_CONNECTION` - `True` - Controls whether the server uses + SSL or not + - `GRAPH_EXP_FETCH_REQUEST_TIMEOUT` - `240000` - Controls the timeout for the + fetch request. Measured in milliseconds (i.e. 240000 is 240 seconds or 4 + minutes). + - `GRAPH_EXP_NODE_EXPANSION_LIMIT` - `None` - Controls the limit for node + counts and expansion queries. +- Conditionally Required: + - Required if `USING_PROXY_SERVER=True` + - `GRAPH_CONNECTION_URL` - `None` - See + [Connections UI](../user-guide/connections.md) + - Required if `USING_PROXY_SERVER=True` and `IAM=True` + - `AWS_REGION` - `None` - See [Connections UI](../user-guide/connections.md) + - `SERVICE_TYPE` - `neptune-db`, Set this as `neptune-db` for Neptune + database or `neptune-graph` for Neptune Analytics. + +### JSON Configuration Approach + +First, create a `config.json` file containing values for the connection +attributes: + +```js +{ + "PUBLIC_OR_PROXY_ENDPOINT": "https://public-endpoint", + "GRAPH_CONNECTION_URL": "https://cluster-cqmizgqgrsbf.us-west-2.neptune.amazonaws.com:8182", + "USING_PROXY_SERVER": true, + "IAM": true, + "SERVICE_TYPE": "neptune-db", + "AWS_REGION": "us-west-2", + // Possible Values are "gremlin", "sparql", "openCypher" + "GRAPH_TYPE": "gremlin", + "GRAPH_EXP_HTTPS_CONNECTION": true, + "PROXY_SERVER_HTTPS_CONNECTION": true, + // Measured in milliseconds (i.e. 240000 is 240 seconds or 4 minutes) + "GRAPH_EXP_FETCH_REQUEST_TIMEOUT": 240000, + "GRAPH_EXP_NODE_EXPANSION_LIMIT": 500, +} +``` + +Pass the `config.json` file path to the `docker run` command. + +```bash +docker run -p 80:80 -p 443:443 \ + --env HOST={hostname-or-ip-address} \ + -v /path/to/config.json:/graph-explorer/config.json \ + public.ecr.aws/neptune/graph-explorer +``` + +### Environment Variable Approach + +Provide the desired connection variables directly to the `docker run` command, +as follows: + +```bash +docker run -p 80:80 -p 443:443 \ + --env HOST={hostname-or-ip-address} \ + --env PUBLIC_OR_PROXY_ENDPOINT=https://public-endpoint \ + --env GRAPH_TYPE=gremlin \ + --env USING_PROXY_SERVER=true \ + --env IAM=false \ + --env GRAPH_CONNECTION_URL=https://cluster-cqmizgqgrsbf.us-west-2.neptune.amazonaws.com:8182 \ + --env AWS_REGION=us-west-2 \ + --env SERVICE_TYPE=neptune-db \ + --env PROXY_SERVER_HTTPS_CONNECTION=true \ + --env GRAPH_EXP_FETCH_REQUEST_TIMEOUT=240000 \ + --env GRAPH_EXP_NODE_EXPANSION_LIMIT=500 \ + public.ecr.aws/neptune/graph-explorer +``` diff --git a/docs/deployment/docker.md b/docs/deployment/docker.md new file mode 100644 index 000000000..b41efc055 --- /dev/null +++ b/docs/deployment/docker.md @@ -0,0 +1,175 @@ +# Docker Deployment + +The quickest way to get started with Graph Explorer is to use the official +Docker image. You can find the latest version of the image on +[Amazon's ECR Public Registry](https://gallery.ecr.aws/neptune/graph-explorer). + + +> [!NOTE] +> +> Make sure to use the version of the image that does not include `sagemaker` in the tag. + +## Prerequisites + +- [Docker](https://docs.docker.com/get-docker/) installed on your machine +- [AWS CLI](https://docs.aws.amazon.com/cli/latest/userguide/install-cliv2.html) + installed on your machine + +## Steps + +1. Authenticate with the Amazon ECR Public Registry. + [More information](https://docs.aws.amazon.com/AmazonECR/latest/public/public-registries.html#public-registry-auth) + + ``` + aws ecr-public get-login-password --region us-east-1 | docker login --username AWS --password-stdin public.ecr.aws + ``` + +2. Pull down the docker image + + ``` + docker pull public.ecr.aws/neptune/graph-explorer + ``` + +3. Create and run a docker container using the image + + ``` + docker run -p 80:80 -p 443:443 \ + --env HOST=localhost \ + --name graph-explorer \ + public.ecr.aws/neptune/graph-explorer + ``` + + The `HOST` environment variable is used for SSL certificates generation since + HTTPS is the default. If you are hosting this on a public domain, you should + replace `HOST=localhost` with your domain name. + +4. Open a browser and type in the URL of the Graph Explorer server instance + + ``` + https://localhost/explorer + ``` + +5. You will receive a warning as the SSL certificate used is self-signed. Since + the application is set to use HTTPS by default and contains a self-signed + certificate, you will need to add the Graph Explorer certificates to the + trusted certificates directory and manually trust them. See the + [HTTPS Connections](https-setup.md#https-connections) section. +6. After completing the trusted certification step and refreshing the browser, + you should now see the Connections UI. See below description on Connections + UI to configure your first connection to Amazon Neptune. + +## Gremlin Server Database + +Gremlin Server is an easy way to get started with graph databases. This example +will configure a simple Gremlin Server instance to be used with Graph Explorer. +It comes with a very small graph dataset. + +1. Pull the latest Gremlin Server image from Docker Hub. + ``` + docker pull tinkerpop/gremlin-server:latest + ``` +2. Create and run the Gremlin Server container using the HTTP REST modern + configuration. + ``` + docker run -p 8182:8182 \ + --name gremlin-server \ + tinkerpop/gremlin-server:latest \ + conf/gremlin-server-rest-modern.yaml + ``` +3. Open Graph Explorer and add a new connection + - Name: `Gremlin Server` + - Graph Type: `Gremlin` + - Public or Proxy Endpoint: `https://localhost` + - Using Proxy Server: `true` + - Graph Connection URL: `http://localhost:8182` + +## Troubleshooting + +### Container Startup Issues + +1. If the container does not start, or immediately stops, use + `docker logs graph-explorer` to check the container console logs for any + related error messages that might provide guidance on why graph-explorer did + not start. +2. If you are having issues connecting graph-explorer to your graph database, + use your browser's Developer Tools feature to monitor both the browser + console and network calls to determine if here are any errors related to + connectivity. + +### Ports in Use + +If the default ports of 80 and 443 are already in use, you can use the `-p` flag +to change the ports to something else. The host machine ports are the first of +the two numbers. You can change those to whatever you want. + +For example, if you want to use port 8080 and 4431, you can use the following +command: + +``` +docker run -p 8080:80 -p 4431:443 \ + --name graph-explorer \ + --env HOST=localhost \ + public.ecr.aws/neptune/graph-explorer +``` + +Which will result in the following URLs: + +- HTTPS: `https://localhost:4431/explorer` +- HTTP: `http://localhost:8080/explorer` + +### HTTP Only + +If you do not want to use SSL and HTTPS, you can disable it by setting the +following [environment variables](configuration.md): + +``` +PROXY_SERVER_HTTPS_CONNECTION=false +GRAPH_EXP_HTTPS_CONNECTION=false +``` + +These can be passed when creating the Docker container like so: + +``` +docker run -p 80:80 \ + --name graph-explorer \ + --env PROXY_SERVER_HTTPS_CONNECTION=false \ + --env GRAPH_EXP_HTTPS_CONNECTION=false \ + public.ecr.aws/neptune/graph-explorer +``` + +### HTTPS Connections + +If either of the Graph Explorer or the proxy-server are served over an HTTPS +connection (which it is by default), you will have to bypass the warning message +from the browser due to the included certificate being a self-signed +certificate. + +You can bypass by manually ignoring them from the browser or downloading the +correct certificate and configuring them to be trusted. Alternatively, you can +provide your own certificate. + +The following instructions can be used as an example to bypass the warnings for +Chrome, but note that different browsers and operating systems will have +slightly different steps. + +1. Download the certificate directly from the browser. For example, if using + Google Chrome, click the "Not Secure" section on the left of the URL bar and + select "Certificate is not valid" to show the certificate. Then click Details + tab and click Export at the bottom. +2. Once you have the certificate, you will need to trust it on your machine. For + MacOS, you can open the Keychain Access app. Select System under System + Keychains. Then go to File > Import Items... and import the certificate you + downloaded in the previous step. +3. Once imported, select the certificate and right-click to select "Get Info". + Expand the Trust section, and change the value of "When using this + certificate" to "Always Trust". +4. You should now refresh the browser and see that you can proceed to open the + application. For Chrome, the application will remain "Not Secure" due to the + fact that this is a self-signed certificate. If you have trouble accessing + Graph Explorer after completing the previous step and reloading the browser, + consider running a docker restart command and refreshing the browser again. + + +> [!TIP] +> To get rid of the "Not Secure" warning, see +[Using self-signed certificates on Chrome](../development/development-setup.md#using-self-signed-certificates-on-chrome). diff --git a/docs/deployment/https-setup.md b/docs/deployment/https-setup.md new file mode 100644 index 000000000..1e8742907 --- /dev/null +++ b/docs/deployment/https-setup.md @@ -0,0 +1,44 @@ +# HTTPS Setup + +This guide covers HTTPS configuration and certificate management for Graph +Explorer deployments. + +## HTTPS Configuration + +Graph Explorer supports the HTTPS protocol by default and provides a self-signed +certificate as part of the Docker image. You can choose to use HTTP instead by +changing the [environment variable default settings](configuration.md). + +## HTTPS Connections + +If either Graph Explorer or the proxy-server are served over an HTTPS connection +(which it is by default), you will have to bypass the warning message from the +browser due to the included certificate being a self-signed certificate. You can +bypass by manually ignoring them from the browser or downloading the correct +certificate and configuring them to be trusted. Alternatively, you can provide +your own certificate. The following instructions can be used as an example to +bypass the warnings for Chrome, but note that different browsers and operating +systems will have slightly different steps. + +1. Download the certificate directly from the browser. For example, if using + Google Chrome, click the "Not Secure" section on the left of the URL bar and + select "Certificate is not valid" to show the certificate. Then click Details + tab and click Export at the bottom. +2. Once you have the certificate, you will need to trust it on your machine. For + MacOS, you can open the Keychain Access app. Select System under System + Keychains. Then go to File > Import Items... and import the certificate you + downloaded in the previous step. +3. Once imported, select the certificate and right-click to select "Get Info". + Expand the Trust section, and change the value of "When using this + certificate" to "Always Trust". +4. You should now refresh the browser and see that you can proceed to open the + application. For Chrome, the application will remain "Not Secure" due to the + fact that this is a self-signed certificate. If you have trouble accessing + Graph Explorer after completing the previous step and reloading the browser, + consider running a docker restart command and refreshing the browser again. + + +> [!TIP] +> +> To get rid of the "Not Secure" warning, see +[Using self-signed certificates on Chrome](../development/development-setup.md#using-self-signed-certificates-on-chrome). diff --git a/docs/development/README.md b/docs/development/README.md new file mode 100644 index 000000000..8f5c5692a --- /dev/null +++ b/docs/development/README.md @@ -0,0 +1,13 @@ +# Development + +This section contains guides for developers who want to contribute to Graph +Explorer or set up a local development environment. + +## Files in this section + +1. [Development Setup](development-setup.md) - Local development environment + setup +2. [Release Preparation](release.md) - Guide for preparing and releasing new + versions +3. [Troubleshooting](troubleshooting.md) - Common development issues and + solutions diff --git a/additionaldocs/development.md b/docs/development/development-setup.md similarity index 54% rename from additionaldocs/development.md rename to docs/development/development-setup.md index a9448f205..619a3d354 100644 --- a/additionaldocs/development.md +++ b/docs/development/development-setup.md @@ -1,15 +1,14 @@ -## Development +# Local Development Setup -This developer README details instructions for building on top of the -graph-explorer application, or for configuring advanced settings, like using -environment variables to switch to HTTP. +This guide details instructions for building on top of the graph-explorer +application, or for configuring advanced settings for development. -### Requirements +## Requirements - pnpm >=10.12.1 - node >=24.4.0 -#### Node Version +### Node Version Ensure you are running the correct Node version. If you are using [NVM](https://github.com/nvm-sh/nvm), you can simply do: @@ -21,7 +20,7 @@ nvm use Otherwise, use whatever method you use to install [Node v24.4.0](https://nodejs.org/en/download). -#### Node Corepack +### Node Corepack [Corepack](https://nodejs.org/api/corepack.html) is used to ensure the package manager used for the project is consistent. @@ -30,12 +29,12 @@ manager used for the project is consistent. corepack enable ``` -### Supported Graph Types +## Supported Graph Types - Labelled Property Graph (PG) using Gremlin or openCypher - Resource Description Framework (RDF) using SPARQL -### Run in development mode +## Run in development mode Install any missing or updated dependencies. @@ -58,7 +57,7 @@ http://localhost:5173 At this point, Graph Explorer should be successfully running and it is asking you for connection details. This part is specific to your personal setup. -### Build for production +## Build for production Building Graph Explorer is simple. @@ -87,7 +86,7 @@ However, if you want to run Graph Explorer without the proxy server, you can: pnpm start:client ``` -### Managing dependencies +## Managing dependencies If you need to add, remove, or update a dependency you can easily do so from the root folder in the CLI: @@ -100,99 +99,36 @@ pnpm add react --filter graph-explorer pnpm add -D vitest --filter graph-explorer-proxy-server ``` -#### Preparation of a release - -This repository is composed by 2 packages and a mono-repository structure -itself. Then, you need to take into account 3 different `package.json` files: - -- `/package.json` is intended to keep the dependencies for managing the - repository. It has utilities like linter, code formatter, or git checks. -- `/packages/graph-explorer/package.json` is the package file that - describes the UI client package. -- `/packages/graph-explorer-proxy-server/package.json` is the package file - for the node server which is in charge of authentication and redirection of - requests. - -Each of these `package.json` files has an independent `version` property. -However, in this project we should keep them correlated. Therefore, when a new -release version is being prepared, the version number should be increased in all -3 files. Regarding the version number displayed in the user interface, it is -specifically extracted from the `/packages/graph-explorer/package.json`. -file - -### Environment variables - -#### `GRAPH_EXP_ENV_ROOT_FOLDER` - -Base path used to serve the `graph-explorer` front end application. - -Example: `/explorer` - -- Optional -- Default: `/` -- Type: `string` - -#### `GRAPH_EXP_CONNECTION_NAME` - -Default connection name. - -- Optional -- Default is empty -- Type: `string` - -#### `GRAPH_EXP_CONNECTION_ENGINE` - -The query engine to use for the default connection. - -- Optional -- Default is `gremlin` -- Valid values are `gremlin`, `openCypher`, or `sparql` -- Type: `string` - -#### `HOST` - -The public hostname of the server. This is used to generate the SSL certificate -during the Docker build. - -Example: `localhost` - -- Required when using HTTPS connections -- Default is empty -- Type: `string` - -#### `GRAPH_EXP_HTTPS_CONNECTION` - -Uses the self-signed certificate to serve Graph Explorer over https if true. - -- Optional -- Default `true` -- Type: `boolean` - -#### `PROXY_SERVER_HTTPS_PORT` - -The port to use for the HTTPS server. - -- Optional -- Default `443` -- Type: `number` - -#### `PROXY_SERVER_HTTP_PORT` - -The port to use for the HTTP server. - -- Optional -- Default `80` -- Type: `number` - -#### `PROXY_SERVER_HTTPS_CONNECTION` - -Uses the self-signed certificate to serve the proxy-server over https if true. - -- Optional -- Default `true` -- Type: `boolean` - -### Using self-signed certificates with Docker +## Building Docker Image from Source + +You can build the Docker image locally by following the steps below. + +1. Clone the repository + ``` + git clone https://github.com/aws/graph-explorer.git + ``` +2. Navigate to the repository + ``` + cd graph-explorer + ``` +3. Build the image + ``` + docker build -t graph-explorer . + ``` +4. Run the container (HTTPS disabled) + ``` + docker run -p 80:80 \ + --name graph-explorer \ + --env PROXY_SERVER_HTTPS_CONNECTION=false \ + --env GRAPH_EXP_HTTPS_CONNECTION=false \ + graph-explorer + ``` +5. Connect to the Graph Explorer UI + ``` + http://localhost/explorer + ``` + +## Using self-signed certificates with Docker - Self-signed certificates will use the hostname provided in the `docker run` command, so unless you have specific requirements, there are no extra steps @@ -208,15 +144,15 @@ Uses the self-signed certificate to serve the proxy-server over https if true. /packages/graph-explorer-proxy-server/cert-info/ on the Docker container that is created. -### Using Self-signed certificates on Chrome +## Using Self-signed certificates on Chrome For browsers like Safari and Firefox, -[trusting the certificate from the browser](../README.md/#https-connections) is -enough to bypass the “Not Secure” warning. However, Chrome treats self-signed +[trusting the certificate from the browser](../deployment/https-setup.md#https-connections) +is enough to bypass the "Not Secure" warning. However, Chrome treats self-signed certificates differently. If you want to use a self-signed certificate on Chrome -**without** the “Not Secure” warning and you do not have your own certificate, -or one provided by Let’s Encrypt, you can use the following instructions to add -the root certificate and remove the warning. These instructions assume you’re +**without** the "Not Secure" warning and you do not have your own certificate, +or one provided by Let's Encrypt, you can use the following instructions to add +the root certificate and remove the warning. These instructions assume you're using an EC2 instance to run the Docker container for Graph Explorer. 1. After the Docker container is built and running, open a terminal prompt and @@ -230,7 +166,7 @@ using an EC2 instance to run the Docker container for Graph Explorer. `scp -i {path_to_pem_file} {EC2_login}:~/rootCA.crt {path_on_local_to_place_file}` For example, `scp -i /Users/user1/EC2.pem ec2-user@XXX.XXX.XXX.XXX:~/rootCA.crt /Users/user1/downloads` -6. After copying the certificate from the container to your local machine’s file +6. After copying the certificate from the container to your local machine's file system, you can delete the rootCA.crt file from the EC2 file store with `rm -rf ~/rootCA.crt` 7. Once you have the certificate, you will need to trust it on your machine. For @@ -241,17 +177,7 @@ using an EC2 instance to run the Docker container for Graph Explorer. Expand the Trust section, and change the value of "When using this certificate" to "Always Trust". 9. You should now refresh the browser and see that you can proceed to open the - application. For Chrome, the application will remain “Not Secure” due to the + application. For Chrome, the application will remain "Not Secure" due to the fact that this is a self-signed certificate. If you have trouble accessing Graph Explorer after completing the previous step and reloading the browser, consider running a docker restart command and refreshing the browser again. - -### Troubleshooting - -- If you need more detailed logs, you can change the log level from `info` in - the default .env file to `debug`. The logs will begin printing the error's - stack trace. -- If Graph Explorer crashes, you can recreate the container or run `pnpm start` -- If Graph Explorer fails to start, check that the provided endpoint is properly - spelled and that you have access to from the environment you are trying to run - in. If you are in a different VPC, consider VPC Peering. diff --git a/docs/development/release.md b/docs/development/release.md new file mode 100644 index 000000000..8b42adfeb --- /dev/null +++ b/docs/development/release.md @@ -0,0 +1,24 @@ +# Release Preparation + +This guide covers the process for preparing and releasing new versions of Graph +Explorer. + +## Preparation of a release + +This repository is composed by 2 packages and a mono-repository structure +itself. Then, you need to take into account 3 different `package.json` files: + +- `/package.json` is intended to keep the dependencies for managing the + repository. It has utilities like linter, code formatter, or git checks. +- `/packages/graph-explorer/package.json` is the package file that + describes the UI client package. +- `/packages/graph-explorer-proxy-server/package.json` is the package file + for the node server which is in charge of authentication and redirection of + requests. + +Each of these `package.json` files has an independent `version` property. +However, in this project we should keep them correlated. Therefore, when a new +release version is being prepared, the version number should be increased in all +3 files. Regarding the version number displayed in the user interface, it is +specifically extracted from the `/packages/graph-explorer/package.json`. +file diff --git a/docs/development/troubleshooting.md b/docs/development/troubleshooting.md new file mode 100644 index 000000000..9c2733090 --- /dev/null +++ b/docs/development/troubleshooting.md @@ -0,0 +1,14 @@ +# Development Troubleshooting + +This guide covers common issues and solutions when developing with Graph +Explorer. + +## Common Development Issues + +- If you need more detailed logs, you can change the log level from `info` in + the default .env file to `debug`. The logs will begin printing the error's + stack trace. +- If Graph Explorer crashes, you can recreate the container or run `pnpm start` +- If Graph Explorer fails to start, check that the provided endpoint is properly + spelled and that you have access to from the environment you are trying to run + in. If you are in a different VPC, consider VPC Peering. diff --git a/docs/getting-started/README.md b/docs/getting-started/README.md new file mode 100644 index 000000000..13f2e4044 --- /dev/null +++ b/docs/getting-started/README.md @@ -0,0 +1,9 @@ +# Getting Started + +This section helps new users install and configure Graph Explorer for the first +time. + +## Files in this section + +1. [Requirements](requirements.md) - System requirements and minimum recommended + versions diff --git a/docs/getting-started/requirements.md b/docs/getting-started/requirements.md new file mode 100644 index 000000000..216199594 --- /dev/null +++ b/docs/getting-started/requirements.md @@ -0,0 +1,19 @@ +# Requirements + +This guide outlines the system requirements and minimum recommended versions for +Graph Explorer and supported graph databases. + +## Minimum Recommended Versions + +Graph Explorer does not block any particular versions of graph databases, but +the queries used may or may not succeed based on the version of the query +engine. + +For Neptune databases, we recommend +[version 1.2.1.0](https://docs.aws.amazon.com/neptune/latest/userguide/engine-releases-1.2.1.0.html) +or above, which include the summary API and TinkerPop 3.6.2. + +For non-Neptune databases, we recommend at least TinkerPop 3.6. + +For detailed database configuration instructions, see the +[Database Setup](../deployment/database-setup.md) guide. diff --git a/docs/troubleshooting/README.md b/docs/troubleshooting/README.md new file mode 100644 index 000000000..ce6528a03 --- /dev/null +++ b/docs/troubleshooting/README.md @@ -0,0 +1,9 @@ +# Troubleshooting + +This section contains workarounds for common issues and information on how to +diagnose problems with Graph Explorer. + +## Files in this section + +1. [Schema Sync](schema-sync.md) - Schema synchronization failures and + connection problems diff --git a/docs/troubleshooting/schema-sync.md b/docs/troubleshooting/schema-sync.md new file mode 100644 index 000000000..3404705d7 --- /dev/null +++ b/docs/troubleshooting/schema-sync.md @@ -0,0 +1,82 @@ +# Schema Sync Issues + +This guide covers common issues when performing schema synchronization in Graph +Explorer. + +## Common Schema Sync Failures + +Schema sync can fail for many reasons. Below are a few of the common ones. + +## Mismatched Proxy Server URL + +When running a schema sync, a common cause of failures is a mismatch between the +proxy server URL in your connection settings and the URL currently loaded in +your browser. This mismatch can trigger a "Connection Error" message in Graph +Explorer. + +Modern browsers enforce a security policy called +[Same-Origin Policy](https://developer.mozilla.org/en-US/docs/Web/Security/Same-origin_policy), +which requires API requests to be sent to the same domain as the page you're +viewing. If your connection settings specify a different domain than your +current browser URL, the request will be blocked. + +To resolve this issue: + +1. Check your connection settings +2. Ensure the proxy server URL matches the domain portion of your current + browser URL +3. Update the connection if necessary + +For example: + +``` +# if the current browser page URL is +https://graph-explorer.mydomain.com/explorer/#/connections + +# the connection setting's proxy server URL should be +https://graph-explorer.mydomain.com +``` + +## Timeout + +There are multiple sources of timeouts. + +- Database server +- Networking layer (load balancer, proxy, etc) +- Browser +- Graph Explorer connection configuration + +The error you receive in Graph Explorer can interpret Neptune timeout errors and +timeouts from Graph Explorer's configuration. + +## Out of Memory + +This can happen when your database is very large. Graph Explorer does its best +to support larger databases and is always improving. Please +[file an issue](https://github.com/aws/graph-explorer/issues/new/choose) if you +encounter this situation. + +## Proxy Server Cannot Be Reached + +Communication between the client and proxy server can be configured in different +ways. When Graph Explorer proxy server starts up it will print out its best +approximation of the correct public proxy server address. + +This can manifest as different types of errors depending on the root cause. You +may receive 404 not found responses or get connection refused errors. + +- The proxy server can be hosting HTTP or HTTPS +- The port of the proxy server could be the default (i.e. 80 or 443 with SSL) or + a specific port provided through environment values +- The proxy server paths are not exposed by the networking layer (load balancer, + proxy, firewall, etc) + - The client is hosted at `/explorer` by default, which is configurable + - Queries are handled via `/gremlin`, `/opencypher`, `/sparql` + - Summary APIs are handled via `/summary`, `/pg/statistics/summary`, + `/rdf/statistics/summary` + - Logging is handled by `/logger` + - Default connection is handled by `/defaultConnection` + +> [!IMPORTANT] +> The paths listed here could always change in the future. If they do change, we +> will note that in the release notes. diff --git a/docs/user-guide/README.md b/docs/user-guide/README.md new file mode 100644 index 000000000..9c91f1d43 --- /dev/null +++ b/docs/user-guide/README.md @@ -0,0 +1,14 @@ +# User Guide + +This section contains guides for end users on how to use Graph Explorer's +features and connect to different graph databases. + +## Files in this section + +1. [Connections](connections.md) - Managing database connections in the UI +2. [Security](security.md) - Security considerations, authentication, and + permissions +3. [Settings](settings.md) - Application settings and configuration management +4. [Graph Explorer](graph-explorer.md) - Main graph visualization interface +5. [Table View](table-view.md) - Tabular data view and export functionality +6. [Data Explorer](data-explorer.md) - Node type data exploration diff --git a/docs/user-guide/connections.md b/docs/user-guide/connections.md new file mode 100644 index 000000000..2614a8a69 --- /dev/null +++ b/docs/user-guide/connections.md @@ -0,0 +1,63 @@ +# Connections UI + +This guide covers how to manage connections within the Graph Explorer +application. For information on configuring different graph database engines, +see the [Database Setup](../deployment/database-setup.md) guide. + +You can create and manage connections to graph databases using this feature. +Connections is accessible as the first screen after deploying the application, +when you click `Open Connections` on the top-right. Click `+` on the top-right +to add a new connection. You can also edit and delete connections. + +### Add a new connection + +- **Name:** Enter a name for your connection (e.g., `MyNeptuneCluster`). +- **Graph Type:** Choose a graph data model that corresponds to your graph + database. +- **Public or proxy endpoint:** Provide the publicly accessible endpoint URL for + a graph database, e.g., Gremlin Server. If connecting to Amazon Neptune, then + provide a proxy endpoint URL that is accessible from outside the VPC, e.g., + EC2. + - **Note:** For connecting to Amazon Neptune, ensure that the graph connection + URL is in the format `https://[NEPTUNE_ENDPOINT]:8182`, and that the proxy + endpoint URL is either `https://[EC2_PUBLIC_HOSTNAME]:443` or + `http://[EC2_PUBLIC_HOSTNAME]:80`, depending on the protocol used. Ensure + that you don't end either of the URLs with `/`. +- **Using proxy server:** Check this box if using a proxy endpoint. +- **Graph connection URL:** Provide the endpoint for the graph database +- **AWS IAM Auth Enabled:** Check this box if connecting to Amazon Neptune using + IAM Auth and SigV4 signed requests +- **Service Type:** Choose the service type +- **AWS Region:** Specify the AWS region where the Neptune cluster is hosted + (e.g., us-east-1) +- **Fetch Timeout:** Specify the timeout for the fetch request +- **Neighbor Expansion Limit:** Specify the default limit for neighbor + expansion. This will override the app setting for neighbor expansion. + +### Available Connections + +Once a connection is created, this section will appear as a left-hand pane. When +you create more than one connection to a graph database, you can only connect to +and visualize from one graph database endpoint at a time. To select the active +database, toggle the "Active" switch. + +### Connection Details + +Once a connection is created, this section will appear as a right-hand +information pane for a selected connection. It shows details such as the +connection name, graph data model type, endpoint and a summary of the graph +data, such as the count of nodes, edges, and a list of node types. + +### Last Synchronization + +When a connection is created, Graph Explorer will perform a scan of the graph to +provide summary data. To re-synchronize after data has changed on your graph, +select a connection, and then click the "refresh" button next to "Last +Synchronization" text. + +### Data Explorer UI + +Under a listed node type, you can click on the '>' arrow to get to the "Data +Explorer" view. This allows you to see a sample list of nodes under this type +and choose one or more nodes to "Send to Explorer" for getting started quickly +if you are new to the data. diff --git a/docs/user-guide/data-explorer.md b/docs/user-guide/data-explorer.md new file mode 100644 index 000000000..cee93e401 --- /dev/null +++ b/docs/user-guide/data-explorer.md @@ -0,0 +1,11 @@ +# Data Explorer UI + +You can use the Data Explorer UI to view the data for the selected node type. +You can open the Data Explorer by clicking the node type row in the connection +details pane. + +## Features + +- View tabular data for the selected node type +- Set the node type display name and description attributes +- Send a specific node to the graph view diff --git a/docs/user-guide/graph-explorer.md b/docs/user-guide/graph-explorer.md new file mode 100644 index 000000000..58948c99e --- /dev/null +++ b/docs/user-guide/graph-explorer.md @@ -0,0 +1,129 @@ +# Graph Explorer UI + +You can search, browse, expand, customize views of your graph data using Graph +Explorer, which is the main UI of this application. Once you create a +connection, you can click "Open Graph Explorer" on the top-right to navigate +here. There are several key features on this UI: + +## Top Bar UI + +- **Toggles:** You can toggle to show/hide the Graph View and/or Table View for + screen real-estate management. +- **Open Connections:** This takes the user back to Connections UI. + +## Graph View UI + +The graph visualization canvas that you can interact with. Double-click to +expand the first-order neighbors of a node. + +- **Layout drop-down & reset:** You can display graph data using standard graph + layouts in the Graph View. You can use the circular arrow to reset the physics + of a layout. +- **Screenshot:** Download a picture of the current window in Graph View. +- **Save Graph:** Save the current rendered graph as a JSON file that can be + shared with others having the same connection or reloaded at a later time. +- **Load Graph:** Load a previously saved graph from a JSON file. +- **Zoom In/Out & Clear:** To help users quickly zoom in/out or clear the whole + canvas in the Graph View. +- **Legend (i):** This displays an informational list of icons, colors, and + display names available. + +## Sidebar Panel UI + +The panel on the right of the graph provides various actions, configuration, and +details about the open graph. + +- [**Search panel**](#search-panel) allows you to search for specific nodes by + filtering on node types & attributes or executing a database query then adding + nodes & edges to the graph panel. +- [**Details panel**](#details-panel) shows details about a selected node/edge + such as properties etc. +- [**Entities filter panel**](#entities-filter-panel) is used to control the + display of nodes and edges that are already expanded in the Graph View; click + to hide or show nodes/edges. +- [**Expand panel**](#expand-panel) provides controls and filters to help focus + large neighbor expansions. +- [**Node styling panel**](#node-styling-panel) of node display options (e.g., + color, icon, the property to use for the displayed name). +- [**Edge styling panel**](#edge-styling-panel) of edge display options (e.g., + color, icon, the property to use for the displayed name). +- [**Namespaces panel (RDF only)**](#namespace-panel) allows you to shorten the + display of Resource URIs within the app based on auto-generated prefixes, + commonly-used prefix libraries, or custom prefixes set by the user. Order of + priority is set to Custom > Common > Auto-generated. + +### Search Panel + +Graph Explorer Search UI provides two powerful ways to search and interact with +your graph database: + +#### Filter Search + +- Enables faceted filtering of nodes based on: + - Node labels (or rdf:type for RDF databases) + - Node attribute values +- Supports partial text matching +- Search results can be added to the graph individually or all at once +- Supports cancellation of long-running queries + +#### Query Search + +- Available for Gremlin and openCypher connections +- Allows execution of any valid database query, including mutations +- When adding an edge, its connected nodes are automatically included +- Displays scalar values in results (though these cannot be added to the graph) +- Paginates large result sets +- Results can be added to the graph individually or all at once +- Supports cancellation of long-running queries + +### Details Panel + +- Displays attributes and values for node or edge selections +- Displays neighbor counts by label for node selections +- Displays relationship information for edge selections + +### Expand Panel + +Provides fine grained control over neighbor expansions + +- Filter by node label (or rdf:type for RDF databases) +- Filter by attribute value +- Limit results to a maximum size + +### Entities Filter Panel + +Provides the ability to filter nodes or edges from the visualization by label +(or rdf:type for RDF databases) + +### Node Styling Panel + +Each node type can be customized in a variety of ways. + +- **Display label** allows you to change how the node label (or rdf:type) is + represented +- **Display name attribute** allows you to choose the attribute on the node that + is used to uniquely label the node in the graph visualization and search +- **Display description attribute** allows you to choose the attribute on the + node that is used to describe the node in search +- **Custom symbol** can be uploaded in the form of an SVG icon +- **Colors and borders** can be customized to visually distinguish from other + node types + +### Edge Styling Panel + +Each edge type can be customized in a variety of ways. + +- **Display label** allows you to change how the edge label (or predicate in RDF + databases) is represented +- **Display name attribute** allows you to choose the attribute on the edge that + is used to uniquely label the edge in the graph visualization and search +- **Arrow symbol** can be chosen for both source and target variations +- **Colors and borders** can be customized for the edge label and the line +- **Line style** can be solid, dotted, or dashed + +### Namespace Panel + +- Only visible in RDF connections +- Displays any automatically generated namespace prefixes for the connection +- Displays all the common prefixes that are built-in +- Allows creation and management of any custom namespace prefixes diff --git a/docs/user-guide/security.md b/docs/user-guide/security.md new file mode 100644 index 000000000..64553007b --- /dev/null +++ b/docs/user-guide/security.md @@ -0,0 +1,39 @@ +# Security + +This guide covers authentication methods and permissions when using Graph +Explorer with graph databases. + +For HTTPS configuration and certificate setup, see the +[HTTPS Setup](../deployment/https-setup.md) guide. + +## Permissions + +Graph Explorer does not provide any mechanisms for controlling user permissions. +If you are using Graph Explorer with AWS, Neptune permissions can be controlled +through IAM roles. + +For information about what permissions Graph Explorer requires check out the +documentation on +[SageMaker configuration](../deployment/aws-sagemaker.md#minimum-database-permissions). + + +> [!CAUTION] +> +> By default, a Neptune Notebook will have full read & write access to Neptune data. + +## Authentication + +Authentication for Amazon Neptune connections is enabled using the +[SigV4 signing protocol](https://docs.aws.amazon.com/general/latest/gr/signature-version-4.html). + +To use AWS IAM authentication, you must run requests through a proxy endpoint, +such as an EC2 instance, where credentials are resolved and where requests are +signed. + +To set up a connection in Graph Explorer UI with AWS IAM auth enabled on +Neptune, check Using Proxy-Server, then check AWS IAM Auth Enabled and type in +the AWS Region where the Neptune cluster is hosted (e.g., us-east-1). + +For further information on how AWS credentials are resolved in Graph Explorer, +refer to this +[documentation](https://docs.aws.amazon.com/AWSJavaScriptSDK/latest/AWS/CredentialProviderChain.html). diff --git a/docs/user-guide/settings.md b/docs/user-guide/settings.md new file mode 100644 index 000000000..dcdb97262 --- /dev/null +++ b/docs/user-guide/settings.md @@ -0,0 +1,22 @@ +# Settings UI + +## General Settings + +- **Default Neighbor Expansion Limit:** This setting will allow you to enable or + disable the default limit applied during neighbor expansion. This applies to + both double click expansion and the expand sidebar. This setting can be + overridden by a similar setting on the connection itself. +- **Save Configuration:** This action will export all the configuration data + within the Graph Explorer local database. This will not store any data from + the connected graph databases. However, the export may contain the shape of + the schema for your databases and the connection URL. +- **Load Configuration:** This action will replace all the Graph Explorer + configuration data you currently have with the data in the provided + configuration file. This is a destructive act and can not be undone. It is + **strongly** suggested that you perform a **Save Configuration** action before + performing a **Load Configuration** action to preserve any existing + configuration data. + +## About + +In the _About_ page you can see the version number and submit any feedback. diff --git a/docs/user-guide/table-view.md b/docs/user-guide/table-view.md new file mode 100644 index 000000000..bef890c7e --- /dev/null +++ b/docs/user-guide/table-view.md @@ -0,0 +1,32 @@ +# Table View UI + +This collapsible view shows a row-column display of the data in the Graph View. +You can use filters in the Table to show/hide elements in the Graph View, and +you can export the table view into a CSV or JSON file. + +## Available Columns + +The following columns are available for filtering on property graphs (RDF graphs +in parentheses): + +- Node ID (Resource URI) +- Node Type (Class) +- Edge Type (Predicate) +- Source ID (Source URI) +- Source Type (Source Class) +- Target ID (Target URI) +- Target Type (Target Class) +- Display Name - Set in the Node/Edge Styling panes +- Display Description - Set in the Node/Edge Styling panes +- Total Neighbors - Enter an integer to be used as the >= limit + +## Additional Table View UI Features + +- **Visibility** - manually show or hide nodes or edges +- **All Nodes / All Edges (or All Resources / All Predicates) dropdown** - + allows you to display a list of either nodes or edges and control + display/filter on them +- **Download** - You can download the current Table View as a CSV or JSON file + with additional customization options +- **Default columns** - You can set which columns you want to display +- Paging of rows