ES详解 - 资料：Awesome Elasticsearch
ES详解 - 资料：Awesome Elasticsearch
本文来自 GitHub Awesome Elasticsearch 项目 在新窗口打开 , 搜集ElasticSearch相关的优秀资料。@pdai
General
Elastic Stack
Elasticsearch 在新窗口打开 official website
Logstash 在新窗口打开 is a data pipeline that helps you process logs and other event data from a variety of systems
Kibana 在新窗口打开 is a data analysis tool that helps to visualize your data; Kibana Manual docs 在新窗口打开
beats 在新窗口打开 is the platform for building lightweight, open source data shippers for many types of data you want to enrich with Logstash, search and analyze in Elasticsearch, and visualize in Kibana.
Books
- teaches you how to leverage neural networks, NLP, and deep learning techniques to improve search performance. (2019)
- demystifies relevance work. Using Elasticsearch, it teaches you how to return engaging search results to your users, helping you understand and leverage the internals of Lucene-based search engines. (2016)
- teaches you how to build scalable search applications using Elasticsearch (2015)
Related (awesome) lists
- frutik/awesome-search 在新窗口打开 I am building e-commerce search now. Below are listed some of my build blocks
Open-source and free products, based on Elasticsearch
- Fess 在新窗口打开 is an open source full featured Enterprise Search, with a web-crawler
- Yelp/elastalert 在新窗口打开 is a modular flexible rules based alerting system written in Python
- etsy/411 在新窗口打开
- an Alert Management Web Application https://demo.fouroneone.io (credentials: user/user)
appbaseio/mirage 在新窗口打开 is a 🔎 GUI for composing Elasticsearch queries
exceptionless/Exceptionless 在新窗口打开 is an error (exceptions) collecting and reporting server with client bindings for a various programming languages
searchkit/searchkit 在新窗口打开 is a UI framework based on React to build awesome search experiences with Elasticsearch
appbaseio/reactivemaps 在新窗口打开 is a React based UI components library for building Airbnb / Foursquare like Maps
appbaseio/reactivesearch 在新窗口打开 is a library of beautiful React UI components for Elasticsearch
appbaseio/dejavu 在新窗口打开 The missing UI for Elasticsearch; landing page 在新窗口打开
Simple File Server 在新窗口打开 is an Openstack Swift compatible distributed object store that can serve and securely store billions of large and small files using minimal resources.
logagent 在新窗口打开 a log shipper to parse and ship logs to Elasticsearch including bulk indexing, disk buffers and log format detection.
ItemsAPI 在新窗口打开 simplified search API for web and mobile (based on Elasticsearch and Express.js)
- An open-source backend with advanced real-time features for Web, Mobile and IoT that uses ElasticSearch as a database. (Website 在新窗口打开 )
- SIAC is an enterprise SIEM built on the ELK stack and other open-source components.
- Sentinl is a Kibana alerting and reporting app.
- Elasticsearch alerting made simple
Elasticsearch developer tools and utilities
Development and debugging
- Sense (from Elastic) 在新窗口打开 A JSON aware developer console to Elasticsearch; official and very powerful
- ES-mode 在新窗口打开 An Emacs major mode for interacting with Elasticsearch (similar to Sense)
- Elasticsearch Cheatsheet 在新窗口打开 Examples for the most used queries, API and settings for all major version of Elasticsearch
- Elasticstat 在新窗口打开 CLI tool displaying monitoring informations like htop
- Elastic for Visual Studio Code 在新窗口打开 An extension for developing Elasticsearch queries like Kibana and Sense extention in Visual Studio Code
- Elastic Builder 在新窗口打开 A Node.js implementation of the Elasticsearch DSL
- Bodybuilder 在新窗口打开 A Node.js elasticsearch query body builder
- enju 在新窗口打开 A Node.js elasticsearch ORM
- Peek 在新窗口打开 An interactive CLI in Python that works like Kibana Console with additional features
Import and Export
- Knapsack plugin 在新窗口打开 is an "swiss knife" export/import plugin for Elasticsearch
- Elasticsearch-Exporter 在新窗口打开 is a command line script to import/export data from Elasticsearch to various other storage systems
- esbulk 在新窗口打开 Parallel elasticsearch bulk indexing utility for the command line.
- elasticdump 在新窗口打开
- tools for moving and saving indices
- Tool for loading common file types to elasticsearch including csv, json, and parquet
Management
- High-level command line interface to manage Elasticsearch clusters.
- Github's open sourced cluster management library based on Elasticsearch's REST API. Comes with a high level CLI tool
Elasticsearch plugins
Cluster
- sscarduzio/elasticsearch-readonlyrest-plugin 在新窗口打开 Safely expose Elasticsearch REST API directly to the public
- mobz/elasticsearch-head 在新窗口打开 is a powerful and essential plugin for managing your cluster, indices and mapping
- Bigdesk 在新窗口打开
- Live charts and statistics for elasticsearch cluster
- Elasticsearch cluster management console with live monitoring and beautiful UI
- Cerebro 在新窗口打开 is an open source(MIT License) elasticsearch web admin tool. Supports ES 5.x
- Kopf 在新窗口打开
- Another management plugin that have REST console and manual shard allocation
- Elasticsearch and elastic stack security and alerting for free
- ee-outliers is a framework to detect outliers in events stored in an Elasticsearch cluster.
- Elasticsearch admin panel built for ops and monitoring
- Web administration for Elasticsearch
Other
- SIREn Join Plugin for Elasticsearch 在新窗口打开 This plugin extends Elasticsearch with new search actions and a filter query parser that enables to perform a "Filter Join" between two set of documents (in the same index or in different indexes).
Integrations and SQL support
- Query elasticsearch using familiar SQL syntax. You can also use ES functions in SQL.
- Elasticsearch real-time search and analytics natively integrated with Hadoop (and Hive)
- JDBC importer for Elasticsearch
- An Elasticsearch client exposing DataFrame API
- Go daemon that syncs MongoDB to Elasticsearch in near realtime
You know, for search
- jprante/elasticsearch-plugin-bundle 在新窗口打开 A plugin that consists of a compilation of useful Elasticsearch plugins related to indexing and searching documents
Kibana plugins and applications
elastic/timelion 在新窗口打开 time-series analyses application. Overview and installation guide: Timelion: The time series composer for Kibana 在新窗口打开
- Kibana plugin with monitoring, alerting and reporting capabilities
- VulnWhisperer is a vulnerability data and report aggregator.
- A Kibana app for working with data generated by Wazuh 在新窗口打开 .
- A real time calculated metric plugin Datasweet Formula 在新窗口打开 .
Kibana Visualization plugins
- a visualization which allows to create live event 3d maps in Kibana
- tag cloud visualization plugin based on d3-cloud plugin
- a plugin for Kibana to view, analyze, search and tail log events from multiple hosts in realtime with devops friendly interface inspired by Papertrail
- Kibana 6 application to manipulate the
_analyzeAPI graphically
- This is a plugin developed for Kibana that displays a network node that link two fields that have been previously selected.
Discussions and social media
Books on Amazon 在新窗口打开 does not fit well into this category, but worth checking out!
TODO: Put some good twitter accounts
Tutorials
- Centralized Logging with Logstash and Kibana On Ubuntu 14.04 在新窗口打开 everything you need to now when you are creating your first Elasticsearch+Logstash+Kibana instance
- dwyl/learn-elasticsearch 在新窗口打开 a getting started tutorial with a pack of valuable references
- Make Sense of your Logs: From Zero to Hero in less than an Hour! by Britta Weber 在新窗口打开 demonstrates how you can build Elasticsearch + Logstash + Kibana stack to collect and discover your data
- $$ Elasticsearch 7 and Elastic Stack 在新窗口打开
- liveVideo course that teaches you to search, analyze, and visualize big data on a cluster with Elasticsearch, Logstash, Beats, Kibana, and more.
Articles
System configuration
A Useful Elasticsearch Cheat Sheet in Times of Trouble 在新窗口打开
The definitive guide for Elasticsearch on Windows Azure 在新窗口打开
9 Tips on Elasticsearch Configuration for High Performance 在新窗口打开
How to Secure Elasticsearch and Kibana 在新窗口打开 with NGINX, LDAP and SSL 🔒
Elasticsearch server on Webfaction using NGINX with basic authorization and HTTPS protocol 在新窗口打开
Elasticsearch Guides 在新窗口打开 Useful Elasticsearch guides with best practices, troubleshooting instructions for errors, tips, examples of code snippets and more.
Docker and Elasticsearch
Java tuning
Elasticsearch Java Virtual Machine settings explained 在新窗口打开
Tuning Garbage Collection for Mission-Critical Java Applications 在新窗口打开
Use Lucene’s MMapDirectory on 64bit platforms, please! 在新窗口打开
G1GC Fundamentals: Lessons from Taming Garbage Collection 在新窗口打开
JVM Garbage Collector settings investigation 在新窗口打开 PDF Comparison of JVM GC
Garbage Collection Settings for Elasticsearch Master Nodes 在新窗口打开 Fine tunine your garbage collector
Understanding G1 GC Log Format 在新窗口打开 To tune and troubleshoot G1 GC enabled JVMs, one must have a proper understanding of G1 GC log format. This article walks through key things that one should know about the G1 GC log format.
How to start using G1
#ES_JAVA_OPTS=""
ES_JAVA_OPTS="-XX:-UseParNewGC -XX:-UseConcMarkSweepGC -XX:+UseG1GC"
Scalable Infrastructure and performance
The Authoritative Guide to Elasticsearch Performance Tuning (Part 1) 在新窗口打开Part 2 在新窗口打开Part 3 在新窗口打开
Tuning data ingestion performance for Elasticsearch on Azure 在新窗口打开
- and not only for Azure. That's a great article about Elasticsearch Performance testing by example
- when you plan to index large amounts of data in Elasticsearch (by Patrick Peschlow)
Elasticsearch for Logging 在新窗口打开 Elasticsearch configuration tips and tricks from Sanity
Scaling Elasticsearch to Hundreds of Developers 在新窗口打开 by Joseph Lynch @yelp
Our Experience of Creating Large Scale Log Search System Using Elasticsearch 在新窗口打开
- topology, separate master, data and search balancers nodes
📂 Elasticsearch on Azure Guidance 在新窗口打开 it is 10% on Azure and 90% of a very valuable general information, tips and tricks about Elasticsearch
How to avoid the split-brain problem in Elasticsearch 在新窗口打开
Datadog's series about monitoring Elasticsearch performance: * How to monitor Elasticsearch performance 在新窗口打开
How to solve 5 Elasticsearch performance and scaling problems 在新窗口打开
Performance Monitoring Essentials - Elasticsearch Edition 在新窗口打开
Integrations
Logging
- 5 Logstash Alternatives 在新窗口打开 and typical use cases
Alerts
- ElastAlert: Alerting At Scale With Elasticsearch, Part 1 在新窗口打开 by engineeringblog.yelp.com
- ElastAlert: Alerting At Scale With Elasticsearch, Part 2 在新窗口打开 by engineeringblog.yelp.com
- Elastalert: implementing rich monitoring with Elasticsearch 在新窗口打开
Time series
- Elasticsearch as a Time Series Data Store 在新窗口打开 by Felix Barnsteiner
- Running derivatives on Voyager velocity data 在新窗口打开 By Colin Goodheart-Smithe
- Shewhart Control Charts via Moving Averages: Part 1 在新窗口打开
- Part 2 在新窗口打开 by Zachary Tong
- Implementing a Statistical Anomaly Detector: Part 1 在新窗口打开
- Part 2 在新窗口打开
- Part 3 在新窗口打开 by Zachary Tong
Machine Learning
- Classifying images into Elasticsearch with DeepDetect 在新窗口打开 (forum thread with discussion 在新窗口打开 ) by Emmanuel Benazera
- Elasticsearch with Machine Learning 在新窗口打开 (English translation 在新窗口打开 ) by Kunihiko Kido
- Recommender System with Mahout and Elasticsearch 在新窗口打开
Use cases for Elasticsearch
- Data Infrastructure at IFTTT 在新窗口打开 Elasticsearch, Kafka, Apache Spark, Redhsift, other AWS services
- OFAC compliance with Elasticsearch 在新窗口打开 using AWS
- Building a Streaming Search Platform 在新窗口打开
- Streaming Search on Tweets: Storm, Elasticsearch, and Redis
Other
- Free and open source log management (based on ES)
- Simple WebUI Monitoring Tool for Logstash ver. 5.0 and up
- An ElasticSearch plugin to return query results as either PDF,HTML or CSV.
- explanations of Elasticsearch terminology, including examples, common best practices and troubleshooting guides for various issues.
Videos
Overviews
Elasticsearch for logs and metrics: A deep dive – Velocity 2016 在新窗口打开 by Sematext Developers
Elasticsearch in action Thijs Feryn 在新窗口打开 a beginner overview
Getting Down and Dirty with ElasticSearch by Clinton Gormley 在新窗口打开
Speed is a Key: Elasticsearch under the Hood 在新窗口打开 introduction + basic performance optimization
$$ Pluralsight: Getting Started With Elasticsearch for .NET Developers 在新窗口打开 this course will introduce users to Elasticsearch, how it works, and how to use it with .NET projects.
$$ Complete Guide to Elasticsearch 在新窗口打开 Comprehensive guide to Elasticsearch, the popular search engine built on Apache Lucene
Scale Your Metrics with Elasticsearch 在新窗口打开 2019 by Philipp Krenn (Elastic) optimization tips and tricks
Advanced
Rafał Kuć - Running High Performance Fault-tolerant Elasticsearch Clusters on Docker 在新窗口打开 and slides 在新窗口打开
Working with Elasticsearch - Search, Aggregate, Analyze, and Scale Large Volume Datastores 在新窗口打开
- O'Reilly Media
- End-to-end Recommender System with Spark and Elasticsearch 在新窗口打开 by Nick Pentreath & Jean-François Puget. Slide deck 在新窗口打开
Code, configuration file samples and other gists
- reyjrar/elasticsearch.yml
- chenryn/ESPL - Elastic Search Processing Language 在新窗口打开 PEG parser sample for SPL to Elasticsearch DSL
- thomaspatzke/EQUEL 在新窗口打开 an Elasticsearch QUEry Language, based on G4 grammar parser
Who is using elasticsearch?
Yelp 在新窗口打开 , IFTTT 在新窗口打开 , StackExchange 在新窗口打开 , Raygun 在新窗口打开 , Mozilla 在新窗口打开 , Spotify 在新窗口打开 , CERN 在新窗口打开 , NASA 在新窗口打开Zalando 在新窗口打开
I want more! (Elasticsearch related resources)
Contributing
- Make sure you are about to post a valuable resource that belongs to this list
- Do NOT group ++Add and --Remove changes in same PR. Make them separate pull requests
- Use spellchecker
- All spelling and grammar corrections are welcome (except for the rule above)
- Fork this repo, do your edits, send the pull request
- Feel free to create any new sections
- Do not even try to add this repo to any awesome-awesome-* lists