Skip to content

Instantly share code, notes, and snippets.

View palash25's full-sized avatar

Palash Nigam palash25

395 followers · 449 following
View GitHub Profile
@palash25
palash25 / bulk_processor_example1.go
Created June 20, 2019 14:24 — forked from olivere/bulk_processor_example1.go
Example #1 of bulk processor usage
// This is an example of using elastic's BulkProcessor with Elasticsearch.
//
// See https://github.com/olivere/elastic and
// and https://github.com/olivere/elastic/wiki/BulkProcessor
// for more details.
/*
* This example illustrates a simple process that performs bulk processing
* with Elasticsearch using the BulkProcessor in elastic.
*
@palash25
palash25 / monitoring_response_query.json
Last active June 15, 2019 15:01
{
"_source": ["_id"],
"from": 1,
"size": 10,
"query": {
"bool": {
"filter": [{
"bool": {
"should": {
"bool": {
@palash25
palash25 / reqBody.json
Created May 26, 2019 16:26
{
"organizationId": "atlan",
"order": "asc",
"allFormsFLag": true,
"formId": "hHhcLKh9st1vFr8nDY4a",
"dateField": "_created_at",
"startDate": "2019-05-20",
"endDate": "2019-05-21",
"userId": [
"gTOAbrbVUkPXgZYNXz2n"
@palash25
palash25 / troubleshoot.md
Last active May 28, 2020 14:32

ES

  • Error: ElasticsearchStatusException[Elasticsearch exception [type=search_phase_execution_exception, reason=all shards failed]]

  • Solution/Reason: Fielddata was disabled

  • Caution: The solution isn't always to turn on fielddata since turning it on will increase the memory consumption by ES (because it creates an inverted index) instead append .keyword to the field name as a workaround.

  • Text fields: These are analyzed fields that can be used for full-text search (exact matches are not needed and docs can be searched using tokens)

  • Keyword fields: These are non-analyzed fields that are used for keyword search (need exact matches to return docs)

@palash25
palash25 / elastic_v3_getting_started.go
Created May 26, 2019 15:47 — forked from olivere/elastic_v3_getting_started.go
Getting started with elastic.v3
package main
import (
"encoding/json"
"fmt"
"reflect"
elastic "gopkg.in/olivere/elastic.v3"
)
@palash25
palash25 / example-bool-query.json
Last active May 24, 2019 13:54
{
"query": {
"bool": {
"filter": [{
"match": {
"client": "android"
}
}, {
"match": {
"isActive": true
@palash25
palash25 / es_contents
Created May 22, 2019 14:01
curl -XPOST 'localhost:9200/response_v2_862875ee3a88a6d09c95bdbda029ce2b/_search?pretty'
{
"took" : 1,
"timed_out" : false,
"_shards" : {
"total" : 1,
"successful" : 1,
"skipped" : 0,
"failed" : 0
},
"hits" : {
@palash25
palash25 / filter-pipeline-comparisons.json
Last active May 21, 2019 13:02
{\"bool\":{\"filter\":{\"FormMatch\":null,\"IsActiveMatch\":null,\"ClientMatch\":null,\"DateRangeMatch\":null,\"UserIdArrayMatch\":null}}}"
{'size': 0, 'query': {'bool': {'filter': [{'match': {'formId': '324fds'}}, {'match': {'isActive': True}}, {'match': {'client': 'android'}}, {'range': {'fff': {'gte': 'fff', 'lte': 'fff'}}}, {'terms': {'_p_createdBy': ['_User$d', '_User$f', '_User$s', '_User$f', '_User$d']}}]}}, 'aggs': {'responses_over_time': {'date_histogram': {'field': 'fdsf', 'interval': 'dsfsdf'}}}}
{'bool': {'filter': [{'match': {'formId': '324fds'}}, {'match': {'isActive': True}}, {'match': {'client': 'android'}}, {'range': {'fff': {'gte': 'fff', 'lte': 'fff'}}}
@palash25
palash25 / es_response.json
Last active May 20, 2019 13:51
{
"response_v2_862875ee3a88a6d09c95bdbda029ce2b" : {
"aliases" : { },
"mappings" : {
"properties" : {
"_created_at" : {
"type" : "date"
},
"_p_createdBy" : {
"type" : "text",
@palash25
palash25 / anahack.py
Created May 19, 2019 17:36
def create_filter_pipeline(body):
"Creates pipeline of filters based on the global filters present on the dashboard"
form_id = body.get("formId")
start_date = body.get("startDate")
end_date = body.get("endDate")
date_field = body.get("dateField", "_created_at")
user_id_array = body.get("userId")
filter_pipeline = []