Результаты исследований: Научные публикации в периодических изданиях › статья › Рецензирование
A statistical test for the Zipf's law by deviations from the Heaps' law. / Chebunin, M. G.; Kovalevskii, A. P.
в: Сибирские электронные математические известия, Том 16, 01.01.2019, стр. 1822-1832.Результаты исследований: Научные публикации в периодических изданиях › статья › Рецензирование
}
TY - JOUR
T1 - A statistical test for the Zipf's law by deviations from the Heaps' law
AU - Chebunin, M. G.
AU - Kovalevskii, A. P.
PY - 2019/1/1
Y1 - 2019/1/1
N2 - We explore a probabilistic model of an artistic text: words of the text are chosen independently of each other in accordance with a discrete probability distribution on an infinite dictionary. The words are enumerated 1, 2,:::, and the probability of appearing the i'th word is asymptotically a power function. Bahadur proved that in this case the number of different words as a function of the length of the text, again, asymptotically behaves like a power function. On the other hand, in the applied statistics community there are statements known as the Zipf's and Heaps' laws that are supported by empirical observations. We highlight the links between Bahadur results and Zipf's/Heaps' laws, and introduce and analyse a corresponding statistical test.
AB - We explore a probabilistic model of an artistic text: words of the text are chosen independently of each other in accordance with a discrete probability distribution on an infinite dictionary. The words are enumerated 1, 2,:::, and the probability of appearing the i'th word is asymptotically a power function. Bahadur proved that in this case the number of different words as a function of the length of the text, again, asymptotically behaves like a power function. On the other hand, in the applied statistics community there are statements known as the Zipf's and Heaps' laws that are supported by empirical observations. We highlight the links between Bahadur results and Zipf's/Heaps' laws, and introduce and analyse a corresponding statistical test.
KW - Heaps' law
KW - Weak convergence
KW - Zipf's law
UR - http://www.scopus.com/inward/record.url?scp=85078653602&partnerID=8YFLogxK
U2 - 10.33048/semi.2019.16.129
DO - 10.33048/semi.2019.16.129
M3 - Article
AN - SCOPUS:85078653602
VL - 16
SP - 1822
EP - 1832
JO - Сибирские электронные математические известия
JF - Сибирские электронные математические известия
SN - 1813-3304
ER -
ID: 23258705