Package org.elasticsearch.index.rankeval

Interface EvaluationMetric

All Superinterfaces:

org.elasticsearch.common.io.stream.NamedWriteable, org.elasticsearch.common.xcontent.ToXContent, org.elasticsearch.common.io.stream.Writeable

All Known Implementing Classes:

DiscountedCumulativeGain, MeanReciprocalRank, PrecisionAtK

public interface EvaluationMetric
extends org.elasticsearch.common.xcontent.ToXContent, org.elasticsearch.common.io.stream.NamedWriteable

Implementations of EvaluationMetric need to provide a way to compute the quality metric for a result list returned by some search (@link SearchHits) and a list of rated documents.

Nested Class Summary

Nested classes/interfaces inherited from interface org.elasticsearch.common.xcontent.ToXContent

org.elasticsearch.common.xcontent.ToXContent.DelegatingMapParams, org.elasticsearch.common.xcontent.ToXContent.MapParams, org.elasticsearch.common.xcontent.ToXContent.Params

Nested classes/interfaces inherited from interface org.elasticsearch.common.io.stream.Writeable

org.elasticsearch.common.io.stream.Writeable.Reader<V extends java.lang.Object>, org.elasticsearch.common.io.stream.Writeable.Writer<V extends java.lang.Object>

Field Summary

Fields inherited from interface org.elasticsearch.common.xcontent.ToXContent

EMPTY_PARAMS

Method Summary

Modifier and Type	Method	Description
default double	combine(java.util.Collection<EvalQueryQuality> partialResults)	how evaluation metrics for particular search queries get combined for the overall evaluation score.
EvalQueryQuality	evaluate(java.lang.String taskId, org.elasticsearch.search.SearchHit[] hits, java.util.List<RatedDocument> ratedDocs)	Returns a single metric representing the ranking quality of a set of returned documents wrt.
static java.util.List<org.elasticsearch.index.rankeval.RatedDocument.DocumentKey>	filterUnknownDocuments(java.util.List<RatedSearchHit> ratedHits)	filter @link RatedSearchHit that don't have a rating
default java.util.Optional<java.lang.Integer>	forcedSearchSize()	Metrics can define a size of the search hits windows they want to retrieve by overwriting this method.
static java.util.List<RatedSearchHit>	joinHitsWithRatings(org.elasticsearch.search.SearchHit[] hits, java.util.List<RatedDocument> ratedDocs)	join hits with rated documents using the joint _index/_id document key

Methods inherited from interface org.elasticsearch.common.io.stream.NamedWriteable

getWriteableName

Methods inherited from interface org.elasticsearch.common.xcontent.ToXContent

isFragment, toXContent

Methods inherited from interface org.elasticsearch.common.io.stream.Writeable

writeTo

Method Detail

evaluate

EvalQueryQuality evaluate(java.lang.String taskId,
                          org.elasticsearch.search.SearchHit[] hits,
                          java.util.List<RatedDocument> ratedDocs)

Returns a single metric representing the ranking quality of a set of returned documents wrt. to a set of document ids labeled as relevant for this search.

Parameters:

taskId - the id of the query for which the ranking is currently evaluated

hits - the result hits as returned by a search request

ratedDocs - the documents that were ranked by human annotators for this query case

Returns:

some metric representing the quality of the result hit list wrt. to relevant doc ids.

joinHitsWithRatings

static java.util.List<RatedSearchHit> joinHitsWithRatings(org.elasticsearch.search.SearchHit[] hits,
                                                          java.util.List<RatedDocument> ratedDocs)

join hits with rated documents using the joint _index/_id document key

filterUnknownDocuments

static java.util.List<org.elasticsearch.index.rankeval.RatedDocument.DocumentKey> filterUnknownDocuments(java.util.List<RatedSearchHit> ratedHits)

filter @link RatedSearchHit that don't have a rating

combine

default double combine(java.util.Collection<EvalQueryQuality> partialResults)

how evaluation metrics for particular search queries get combined for the overall evaluation score. Defaults to averaging over the partial results.

forcedSearchSize

default java.util.Optional<java.lang.Integer> forcedSearchSize()

Metrics can define a size of the search hits windows they want to retrieve by overwriting this method. The default implementation returns an empty optional.

Returns:

the number of search hits this metrics requests