As a popular open-source project, Elasticsearch has a great community and powerful debugging and monitoring tools. Many of the custom features in ThriftDB are available out-of-the box in Elasticsearch. When we decided it was time to replace ThriftDB, Elasticsearch was a natural fit. As Octopart grew, we had performance problems as well: our search index lagged hours behind the document store. It lacked tooling and was difficult to debug. New developers had to understand many layers before making changes.
As internal libraries tend to do, ThriftDB accumulated cruft and hacks over the years. However, Octopart continued to use ThriftDB internally. In 2014, we discontinued and the public ThriftDB service to focus completely on part search. ThriftDB worked well for Octopart over the years and enabled quick iteration on schema design and search implementation. ThriftDB leveraged Thrift definitions to automate the creation of Solr schemas while providing many useful features not available at the time like document storage, dynamic fields, and schema introspection. Solr was the most full featured open-source search engine. When Octopart was founded in 2007, Elasticsearch did not exist yet. Longtime Octopart and HackerNews users may remember and - projects built on Octopart’s ThriftDB technology stack. ThriftDB is an application built at Octopart to keep the Solr search index, Thrift document store, and relational database synchronized while supporting schema changes and multiple document types. While this is a major change to the backend that powers Octopart, on the surface everything looks the sameĪt Octopart we have a search index, which provides ordered results to user queries, a document store, which returns content for the results and detail pages, and a relational database that provides persistent data storage. Recently, Octopart switched from a custom internal library, ThriftDB, to Elasticsearch.Įver worked hard on a project but when you show it off everyone says, “wait, what’s different?” In this post, we’re going to attempt to show off one of those projects.įour weeks ago, we replaced our SolrCloud cluster and custom document store with a shiny new Elasticsearch cluster. Octopart manages structured data for 30 million parts and growing. Today, over 500,000 engineers, scientists, and sourcing professionals use Octopart’s tools to search for parts across thousands of suppliers. It was founded by a trio of experimental physicists with a vision of a better way to find parts online. Octopart is the leading online solutions provider for engineers, maker pros, and component purchasers. Note: This post was originally published on Octopart’s blog on November 4, 2015. Please note that Found is now known as Elastic Cloud.
UPDATE: This article refers to our hosted Elasticsearch offering by an older name, Found.
0 kommentar(er)
