hive sql之 視窗函式。

一、row_number 函式

區別於其它排序函式，相當於多了乙個排序的列。

資料準備：

name money

ming 12

yang 23

ming 35

ming 54

yang 43

1、按照其中money 進行排序並增加一行。

select name,money,

row_number
() over (order by money) from lei_test;
結果：ming    12
1yang    23
2ming    35
3yang    43
4ming    54
5

select name,money ,

row_number()
over
(partition by name order by money) from lei_test;
ming    12
1ming    35
2ming    54
3yang    23
1yang    43
2

select name,money,tb from (select name,money,

row_number
() over (partition by name order by money)
) tb from lei_test ) where tb =1;
結果：ming    12
1yang    23
1

1、rank() 函式和row_number 函式類似就是他會跳過成績相同的資料（如果 按照 name partiton並按照money進行order by 的化他相同的money 就會 row_number 會相同）

例如：

select name、money,

rank
() over (partition by name order by money) from lei_test;
結果為：
ming    12
1ming    12
1ming    35
3ming    54
4yang    23
1yang    43
2

dense_rank 函式和row_number函式類似。（區別：相同成績會並列，序列排序）

select name,money,

dense_rank()
over
(partition by name order by money) from lei_test;
結果ming    12
1ming    12
1ming    35
2ming    54
3yang    23
1yang    43
2

select name,money cume_dist

() over (partition by name order by money) from lei_test;
結果為：
ming    12
0.5ming    12
0.5ming    35
0.75
ming    54
1.0yang    23
0.5yang    43
1.0

select name,money,

percent_rank()
over
(partition by name order by money) from lei_test;
結果為ming    12
0.0ming    12
0.0ming    35
0.6666666666666666
ming    54
1.0yang    23
0.0yang    43
1.0

select name,money ,

ntile(2
) over (partition by name order by money) from lei_test;
結果為：
ming    12
1ming    12
1ming    35
2ming    54
2yang    23
1yang    43
2

1、leag(col，n,default)用於統計視窗內往上的n行值（上面可能只是給個 排序行數 這個返回的值）

select name,money,lead（money,

1）over (partition by name order by money) from lei_test;
返回結果：
ming    12
12ming    12
35ming    35
54ming    54    null
yang    23
43yang    43    null

select name,money,lag(money,1)

over(partition by name order by money)

from lei_test;

2、得到排序後的第乙個值

select name,money first_value

(money) over (partition by name ordre  by money) from lei_test;
返回結果為：
ming    12
12ming    12
12ming    35
12ming    54
12yang    23
23yang    43
23

select name.money,

last_value
(money) over (partition by name order by money) from lei_test:
返回的結果為
ming    12
12ming    12
12ming    35
35ming    54
54yang    23
23yang    43
43

Hive之視窗函式

對一定視窗期內的資料進行聚合。示例 select sum a.pv over partition by cookieid order by create time rows between 3 preceding and current row as pv1,sum a.pv over partit...

hive視窗函式 Hive sql視窗函式原始碼分析

在了解了視窗函式實現原理 spark hive中視窗函式實現原理覆盤 和 sparksql比hivesql優化的點 視窗函式 之後，今天又擼了一遍hive sql 中視窗函式的原始碼實現，寫個筆記記錄一下。簡單來說，視窗查詢有兩個步驟 將記錄分割成多個分割槽 然後在各個分割槽上呼叫視窗函式。傳統的 ...

Scala之視窗函式排序

scala視窗函式這排序rank，重複排序和不重複排序 1 引入包 import org.apache.spark.sql.expressions.window import spark.implicits.2.建立測試的df，可以直接貼上測試。3.選擇按name分組，按score排序，且倒序。va...