The Zipf Mystery

Video Description

Support Vsauce, your brain, Alzheimer's research, and other YouTube educators by joining THE CURIOSITY BOX: a seasonal delivery of viral science toys made by Vsauce! A portion of all proceeds goes to Alzheimer's research and our Inquisitive Fellowship, a program that gives money and resources directly to rising STEM channels here on YouTube! https://www.curiositybox.com http://www.twitter.com/tweetsauce http://www.instagram.com/electricpants WordCount.org http://www.wordcount.org/ How many days have you been alive? http://www.beatcanvas.com/daysalive.asp random letter generator: http://www.dave-reed.com/Nifty/randSeq.html Dictionary of Obscure Sorrows: https://www.youtube.com/user/obscuresorrows Word frequency resources: [lemmatized] https://en.wikipedia.org/wiki/Most_common_words_in_English http://www.uow.edu.au/~dlee/corpora.htm http://www.wordfrequency.info http://www.anc.org/data/anc-second-release/frequency-data/ http://www.titania.bham.ac.uk/docs/ http://www.kilgarriff.co.uk/bnc-readme.html#raw https://en.wiktionary.org/wiki/Wiktionary:Frequency_lists http://ucrel.lancs.ac.uk/bncfreq/ [PDF] http://www.wordfrequency.info/files/entries.pdf [combined Wikipedia and Gutenberg] http://www.monlp.com/2012/04/16/calculating-word-and-n-gram-statistics-from-a-wikipedia-corpora/ http://corpus.byu.edu/coca/files/100k_samples.txt http://corpus.byu.edu/ http://corpus.leeds.ac.uk/list.html https://books.google.co.uk/books?id=ja1_AAAAQBAJ&dq=word+frequency+coca&lr= http://www.ling.helsinki.fi/kit/2009s/clt231/NLTK/book/ch01-LanguageProcessingAndPython.html Great Zipf's law papers: http://colala.bcs.rochester.edu/papers/piantadosi2014zipfs.pdf http://www.ling.upenn.edu/~ycharles/sign708.pdf http://arxiv.org/pdf/cond-mat/0412004.pdf http://www-personal.umich.edu/~mejn/courses/2006/cmplxsys899/powerlaws.pdf Zipf’s law articles and discussions: http://www.theatlantic.com/magazine/archive/2002/04/seeing-around-corners/302471/ http://io9.com/the-mysterious-law-that-governs-the-size-of-your-city-1479244159?utm_expid=66866090-48.Ej9760cOTJCPS_Bq4mjoww.0 https://plus.maths.org/content/os/latestnews/may-aug08/food/index http://judson.blogs.nytimes.com/2009/05/19/math-and-the-city/?em https://plus.maths.org/content/mystery-zipf?src=aop http://www.datasciencecentral.com/profiles/blogs/why-zipf-s-law-explains-so-many-big-data-and-physics-phenomenons https://en.wikipedia.org/wiki/Zipf%27s_law other Zipf’s law PDFs http://ftp.iza.org/dp3928.pdf http://arxiv.org/pdf/1402.2965.pdf http://arxiv.org/pdf/1104.3199.pdf http://www.lel.ed.ac.uk/~jim/zipfjrh.pdf http://www.ncbi.nlm.nih.gov/pmc/articles/PMC2834740/#pone.0009411-Mandelbrot1 http://polymer.bu.edu/hes/articles/pgs02a.pdf in untranslated language: http://arxiv.org/pdf/0808.2904.pdf http://pages.stern.nyu.edu/~xgabaix/papers/zipf.pdf http://www.hpl.hp.com/research/idl/papers/ranking/ranking.html http://statweb.stanford.edu/~owen/courses/306a/ZipfAndGutenberg.pdf http://arxiv.org/pdf/1310.0448v3.pdf http://www.kornai.com/Papers/glotto5.pdf Zipf’s law slides: http://www.slideshare.net/guest9fc47a/nlp-new-words Pareto Principle and related ‘laws’: http://www.squawkpoint.com/2013/03/pareto-principle/ http://billyshall.com/blog/post/paretos-principle https://en.wikipedia.org/wiki/Pareto_principle Random typing and Zipf: http://www.longtail.com/the_long_tail/2006/09/is_zipfs_law_ju.html health 80/20: http://archive.ahrq.gov/research/findings/factsheets/costs/expriach/expriach1.html Principle of least effort: https://en.wikipedia.org/wiki/Principle_of_least_effort https://en.wikipedia.org/wiki/Satisficing http://www.pnas.org/content/100/3/788.full.pdf [PDF] http://csiss.org/classics/content/99 self organized criticality: http://journal.frontiersin.org/article/10.3389/fnsys.2014.00166/full Hapax Legomenon: http://campus.albion.edu/english/2011/02/15/hapax-legomenon/ http://www.dailywritingtips.com/is-that-a-hapax-legomenon/ https://en.wikipedia.org/wiki/Hapax_legomenon [PDF] http://www.aclweb.org/anthology/J10-4003 http://www.wired.com/2012/01/hapax-legomena-and-zipfs-law/ http://oed.hertford.ox.ac.uk/main/content/view/402/450/index.html#_ftn1 http://oed.hertford.ox.ac.uk/main/content/view/36/166/index.html Learning curve: https://en.wikipedia.org/wiki/Learning_curve Forgetting curve: http://www.trainingindustry.com/wiki/entries/forgetting-curve.aspx https://en.wikipedia.org/wiki/Forgetting_curve Experience curve effects: https://en.wikipedia.org/wiki/Experience_curve_effects Forgetting and zipf's law: http://act-r.psy.cmu.edu/wordpress/wp-content/uploads/2012/12/37JRA_LS_PS_1991.pdf http://public.psych.iastate.edu/shacarp/Wixted_Carpenter_2007.pdf http://marshalljonesjr.com/youll-remember-less-than-001-of-your-life/ https://en.wikipedia.org/wiki/Forgetting https://www.reddit.com/r/Showerthoughts/comments/3gu9qk/it_only_takes_three_generations_for_you_to_be/ music from: http://www.youtube.com/jakechudnow http://www.audionetwork.com