about summary refs log tree commit diff stats
path: root/kgramstats.cpp
Commit message (Collapse)AuthorAgeFilesLines
...
* Rewrote weighted random number generatorFeffernoose2013-10-051-33/+37
| | | | | | The previous method of picking which token was the next one was flawed in some mysterious way that ended up picking various words that occurred only once in the input corpus as the first word of the generated output (most notably, "hysterically," "Anarchy," "Yorkshire," and "impunity.").
* Changed incidence of random kgram-trimmingFeffernoose2013-10-041-4/+10
| | | | Also added better terminal output
* Weighed token casing and presence of periodsFeffernoose2013-10-011-25/+67
| | | | | | | | Tokens which differ only by casing or the presence of an ending period are now considered the same token. When tokens are generated, they are cased based on the prevalence of Upper/Title/Lower casing of the token in the input corpus, and similarly, a period is added to the end of the word based on how often the same token was ended with a period in the input corpus.
* Wrote programFeffernoose2013-10-011-0/+110