Commit message (Collapse) | Author | Age | Files | Lines | |
---|---|---|---|---|---|
* | Rewrote how tokens are handled | Kelly Rauchenberger | 2016-01-29 | 1 | -0/+34 |
A 'word' is now an object that contains a distribution of forms that word can take. For now, most word just contain one form, the canonical one. The only special use is currently hashtags. Malapropisms have been disabled because of compatibility issues and because an upcoming feature is planned to replace it. |