1.Two types of rules
Prometheus supports two types of rules .whitch may be configured and then evaluated at regular intervals:
Recording rules
and Alerting rules
.
2.Recording rules
1.Recording rules allow you to precompute
frequently needed
or computationally expensive
expressions and save their result as a new set of time series.
2.Be mutch faster than executing the origin expression every time it’s need.
3.Rules within a group run sequentially at a regular interval.
the syntax of a rule file following


the rule group following


the recording rule following


the alerting rule following


3.Alerting rules
1.Alerting rules allow you to define some alert conditions that used Prometheus expression language and to send notifaction about firing alerts to an external service.
an example of alerting config would like this


and a template description and annotation would like this


3.1 Template
1.Prometheus support templating in annotations and labels of alerts.
2.Some functions such as iterate over data,use conditionals,format data.
3.The prometheus template language is based on the Go Template.
3.1.1 Template examples
Iteration


Display one value


Use console url param


That’s the meaning of: http://xxx.html?instance=127.0.0.1
then the .Params.instance
will evaluate to 127.0.0.1
Advanced iteration


define reusable tempalte


3.1.2 Template reference
go template https://golang.org/pkg/text/template/#hdrFunctions
Queries
Name  Arguments  Returns  Notes 

query  query string  []sample  Queries the database, does not support returning range vectors. 
first  []sample  sample  Equivalent to index a 0 
label  label, sample  string  Equivalent to index sample.Labels label 
value  sample  float64  Equivalent to sample.Value 
sortByLabel  label, []samples  []sample  Sorts the samples by the given label. Is stable. 
first
, label
and value
are intended to make query results easily usable in pipelines.
Numbers
Name  Arguments  Returns  Notes 

humanize  number  string  Converts a number to a more readable format, using metric prefixes. 
humanize1024  number  string  Like humanize , but uses 1024 as the base rather than 1000. 
humanizeDuration  number  string  Converts a duration in seconds to a more readable format. 
humanizePercentage  number  string  Converts a ratio value to a fraction of 100. 
humanizeTimestamp  number  string  Converts a Unix timestamp in seconds to a more readable format. 
Humanizing functions are intended to produce reasonable output for consumption by humans, and are not guaranteed to return the same results between Prometheus versions.
Strings
Name  Arguments  Returns  Notes 

title  string  string  strings.Title, capitalises first character of each word. 
toUpper  string  string  strings.ToUpper, converts all characters to upper case. 
toLower  string  string  strings.ToLower, converts all characters to lower case. 
match  pattern, text  boolean  regexp.MatchString Tests for a unanchored regexp match. 
reReplaceAll  pattern, replacement, text  string  Regexp.ReplaceAllString Regexp substitution, unanchored. 
graphLink  expr  string  Returns path to graph view in the expression browser for the expression. 
tableLink  expr  string  Returns path to tabular (“Table”) view in the expression browser for the expression. 
Others
Name  Arguments  Returns  Notes 

args  []interface{}  map[string]interface{}  This converts a list of objects to a map with keys arg0, arg1 etc. This is intended to allow multiple arguments to be passed to templates. 
tmpl  string, []interface{}  nothing  Like the builtin template , but allows nonliterals as the template name. Note that the result is assumed to be safe, and will not be autoescaped. Only available in consoles. 
safeHtml  string  string  Marks string as HTML not requiring autoescaping. 
4.PromeQL
4.1 operators
complute
+  * / % ^
compare binary
== != >= <= > <
Logical set binary
and , or ,unless
Vector Matching
one to one like this




many to one and one to many like this




Aggreation operators
sum
(calculate sum over dimensions)min
(select minimum over dimensions)max
(select maximum over dimensions)avg
(calculate the average over dimensions)group
(all values in the resulting vector are 1)stddev
(calculate population standard deviation over dimensions)stdvar
(calculate population standard variance over dimensions)count
(count number of elements in the vector)count_values
(count number of elements with the same value)bottomk
(smallest k elements by sample value)topk
(largest k elements by sample value)quantile
(calculate φquantile (0 ≤ φ ≤ 1) over dimensions)
parameter
is only required for count_values
, quantile
, topk
and bottomk
.
Without remove the listed labels from the result vector
By does the opposite and drops labels that are not list in the by
clause




4.2 functions
 abs(): absolute value
 absent():
 ceil(): rounds the sample values of all elements in
v
up to the nearest integer.  changes() :For each input time series,
changes(v rangevector)
returns the number of times its value has changed within the provided time range as an instant vector.  clamp_max(): clamps the sample values of all elements in
v
to have an upper limit ofmax
.  clamp_min(): clamps the sample values of all elements in
v
to have a lower limit ofmin
.  day_of_month(): returns the day of the month for each of the given times in UTC. Returned values are from 1 to 31.
 day_of_week(): returns the day of the week for each of the given times in UTC. Returned values are from 0 to 6, where 0 means Sunday etc.
 day_in_month(): returns number of days in the month for each of the given times in UTC. Returned values are from 28 to 31.
 delta():
 deriv(): calculates the persecond derivative of the time series in a range vector
v
, using simple linear regression.  exp(): calculates the exponential function for all elements in
v
 floor(): rounds the sample values of all elements in
v
down to the nearest integer.  histogram_quantile(): calculates the φquantile (0 ≤ φ ≤ 1) from the buckets
b
of a histogram.  holt_winters():
 hour(): returns the hour of the day for each of the given times in UTC. Returned values are from 0 to 23.
 idelta(): calculates the difference between the last two samples in the range vector
v
, returning an instant vector with the given deltas and equivalent labels.  increase(): calculates the increase in the time series in the range vector. Breaks in monotonicity (such as counter resets due to target restarts) are automatically adjusted for
 irate(): calculates the persecond instant rate of increase of the time series in the range vector.
 label_join():
 label_replace():
 ln(): calculates the natural logarithm for all elements in
v
 log2(): calculates the binary logarithm for all elements in
v
 log10(): calculates the decimal logarithm for all elements in
v
 minute(): returns the minute of the hour for each of the given times in UTC. Returned values are from 0 to 59.
 month(): returns the month of the year for each of the given times in UTC. Returned values are from 1 to 12, where 1 means January etc.
 predict_linear(): predicts the value of time series
t
seconds from now, based on the range vectorv
, using simple linear regression.  rate(): calculates the persecond average rate of increase of the time series in the range vector.
 resets(): returns the number of counter resets within the provided time range as an instant vector.
 round(): rounds the sample values of all elements in
v
to the nearest integer  scalar(): returns the sample value of that single element as a scalar.
 sort(): returns vector elements sorted by their sample values, in ascending order.
 sort_desc():
 sqrt(): calculates the square root of all elements in
v
.  time(): returns the number of seconds since January 1, 1970 UTC. Note that this does not actually return the current time, but the time at which the expression is to be evaluated.
 timestamp(): returns the timestamp of each of the samples of the given vector as the number of seconds since January 1, 1970 UTC.
 vector(): returns the scalar
s
as a vector with no labels.  year(): returns the year for each of the given times in UTC.
<aggregation>_over_time()
:
4.3 demos


4.4 http api
expression queries


range query


metadata


label names


label values


targets


rules


alerts


target metadata


metric metadata


alertmanagers


config


runtime information


build information


over