Category Archives: linguistics

Zipfy n-grams

In the 1930s and 40s George Kingsley Zipf studied word frequencies in several languages and came up with a general observation: If you sort all the words from commonest to rarest, the frequency of the word at rank r is … Continue reading

Posted in computing, linguistics | 10 Comments

The Library of Babble

The new issue of American Scientist is out, both on newsstands and on the web. My “Computing Science” column takes up a topic I’ve already written about here on bit-player: the huge corpus of “n-grams” extracted from the Google Books … Continue reading

Posted in computing, linguistics | 9 Comments

3.14

All those books that Google has been scanning for the past ten years are surprisingly rich in numbers as well as words. The Google Books data set released last December by a Harvard-Google team includes (by my count) 9,620,835,344 occurrences of 458,794 distinct … Continue reading

Posted in linguistics, mathematics | 9 Comments

Googling the lexicon

In 1860, when work began on the New English Dictionary on Historical Principles (better known today as the Oxford English Dictionary), the basic plan was to build an index to all of English literature. Volunteer readers would pore over texts … Continue reading

Posted in books, computing, featured, linguistics | 3 Comments