Background and problem description In late 2016 we were approached with an opportunity to build a data pipeline for one of our large clients.
Performance enhancements with Amazon Neptune
Querying performance enhancements with Neptune versus DynamoDB with OpenSearch.
AWS Neptune and DynamoDB are two highly popular and fully managed databases. They are designed for different use cases and their performance varies depending on the use case.
AWS Neptune is a fully-managed graph database service that is optimized for storing and querying large-scale, highly connected data. It supports popular graph data models, such as Property Graph and W3C's RDF. It also supports popular graph query languages such as Apache TinkerPop Gremlin and W3C’s SPARQL. Neptune can be used for a variety of use cases, including social networking, fraud detection, and recommendation engines. It is designed to scale automatically with continuous backup and replications across Availability Zones.
- High performance and scalability
- High throughput and low latency graph queries
- Built for querying/storing highly connected and complex data
- On-demand charges by hourly usage
DynamoDB, on the other hand, is a simple NoSQL database that is designed to run high-performance applications at scale. It is optimized for low-latency, high-throughput operations. OpenSearch can be integrated with DynamoDB to expand the databases querying capabilities.
- Built for simple tabular data
- Choice between on-demand and provisioned capacity
- Fully managed (serverless)
- Limited querying options
- Difficult to predict cost
Use case on RPGMatch
Both are capable of handling large amounts of data and can be used for a variety of use cases, but the choice between them will depend on the specific requirements of your application. When it comes to querying highly connected and complex data, AWS Neptune outperforms OpenSearch on DynamoDB by a wide margin. Recently the app RPGMatch has started a migration from using OpenSearch on DynamoDB to Neptune to handle querying for users on the platform. The following graphs below were generated using AWS Lambda Power Tools to show the stark increase in performance and reduction in cost when using Neptune for performing queries on RPGMatch user data.
When choosing a database for social networking sites like RPGMatch, that require complex queries, the clear answer is Neptune when it comes to performance, cost, and scalability. It is important to consider the specific requirements of your application and weigh the costs and benefits of each option before making a decision on databases.