summary refs log tree commit diff stats
Commit message (Collapse)AuthorAgeFilesLines
* De-duped pronunciations in generated database hkutilStar Rauchenberger2022-11-302-3/+10
| | | | | | Identical pronunciations will now share an idea and be re-used by multiple forms. This has a negligible effect on database size, but it's useful for writing queries looking for words with the exact same pronunciations. This constitutes a minor database update, which we will call d1.2.
* More hkutil refactoringKelly Rauchenberger2018-09-2723-1920/+866
| | | | | | | | All database access goes through hatkirby::database now. verbly::token, verbly::statement::condition, and verbly::part have been converted to use mpark::variant now. verbly::binding has been deleted, and replaced with a mpark::variant typedef in statement.h. This means that the only remaining tagged union class is verbly::generator::part. refs #5
* Removed unnecessary ROWIDs from database schemaKelly Rauchenberger2018-09-264-80/+92
| | | | | | | | The generator also now sorts and uniq's the WordNet files for antonymy, classification, and pertainymy/mannernymy, because those files contained duplicate rows, and the join tables without ROWIDs now enforce a uniqueness constraint. This constitutes a minor database update -- the new database is compatible with d1.0, but is ~12MB smaller. refs #6
* Re-created VerbNet 3.2 patchKelly Rauchenberger2018-09-262-815/+1609
| | | | The previous file was not formatted in a way that was easy for patch to use. Also, the file has now been renamed to indicate that it is for VerbNet 3.2.
* Replaced some split/implode uses with hkutilKelly Rauchenberger2018-08-106-18/+16
|
* Converted verbly::filter to use a variant objectKelly Rauchenberger2018-04-014-756/+433
|
* Converted asserts in generator to exceptionsKelly Rauchenberger2018-03-316-18/+26
|
* Migrated generator to hkutilKelly Rauchenberger2018-03-3120-262/+331
|
* Started migrating to hkutil (does not build)Kelly Rauchenberger2018-03-3011-659/+105
|
* Added mask filters and fixed the synonym queryKelly Rauchenberger2017-12-214-11/+193
| | | | refs #1
* Created database versioning system d1.0Kelly Rauchenberger2017-11-086-0/+129
| | | | | | | | | | | Also added an ANALYZE statement to the end of the datafile generation process. This generates information that allows sqlite to sometimes come up with a better query plan, and in many cases can significant speed up queries. This constitutes a minor database update, but because this is the first version that uses the database versioning system, older versions are essentially incompatible. refs #2
* Added quote tokenKelly Rauchenberger2017-11-082-7/+39
| | | | This token wraps the inner token in two provided delimiters.
* Renamed definiteArticle token to indefiniteArticleKelly Rauchenberger2017-11-082-14/+16
|
* Fixed bug with token title case transformKelly Rauchenberger2017-10-281-2/+30
| | | | | Word tokens and literal tokens that contained more than one word would only capitalize the first word; this has been fixed.
* Added length field to form tableKelly Rauchenberger2017-10-165-4/+27
| | | | This commit contains a database update.
* Added more casing options to tokensKelly Rauchenberger2017-02-242-21/+80
|
* Added method to check if a token is emptyKelly Rauchenberger2017-02-191-0/+5
|
* Added transform tokensKelly Rauchenberger2017-02-162-5/+244
|
* Added word::synonyms join field (BAD)Kelly Rauchenberger2017-02-162-0/+32
| | | | | | Note that this is not a great implementation; the filter generated is mergable with unrelated filters and may cause results that are misleading.
* Fixed weird filter normalization crashKelly Rauchenberger2017-02-161-13/+13
|
* Tweaked database indexesKelly Rauchenberger2017-02-131-3/+3
| | | | | | | | `rhymes_with` now also contains `prerhyme` so that rhyming joins can be convering. `notions_lemmas` and `lemmas_notions` have been created so as to faciliate "jumping" over `words` when it's only needed as a many-to-many through table. Because `notion_words` and `lemma_words` are prefixes of these new indexes, they have been removed.
* Expanded some indexesKelly Rauchenberger2017-02-111-34/+34
| | | | These modifications can make some queries run significantly faster.
* Fixed statement generation involving two CTEs for the same tableKelly Rauchenberger2017-02-111-2/+10
|
* Added negative filter conversions to objectsKelly Rauchenberger2017-02-105-40/+103
|
* Renamed object validity checksKelly Rauchenberger2017-02-107-7/+7
| | | | | The bool conversion operator was unfortunately Very Confusing so I've just renamed the methods to isValid.
* Made pronunciation::rhymes join dynamicKelly Rauchenberger2017-02-0624-365/+574
| | | | | | | | | | | | | | | | | | | | | | | | | This involved adding a new type of filter; one that compares (currently only equality and inequality) a field with another field located in an enclosing join context. In the process, it was discovered that simplifying the lemma::forms join field earlier actually made some queries return inaccurate results because the inflection of the form was being ignored and anything in the lemma would be used because of the inner join. Because the existing condition join did not allow for the condition field to be on the from side of the join, two things were done: a condition version of joinThrough was made, and lemma was finally eliminated as a top-level object, replaced instead with a condition join between word and form through lemmas_forms. Queries are also now grouped by the first select field (assumed to be the primary ID) of the top table, in order to eliminate duplicates created by inner joins, so that there is a uniform distribution between results for random queries. Created a database index on pronunciations(rhyme) which decreases query time for rhyming filters. The new database version is backwards-compatible because no data or structure changed.
* Fixed error with notion::words joinKelly Rauchenberger2017-02-051-1/+1
|
* Added some missing includesKelly Rauchenberger2017-02-052-0/+2
|
* Flattened selrestrsKelly Rauchenberger2017-02-0520-12732/+468
| | | | | | | | | | | | | | | Now, selrestrs are, instead of logically being a tree of positive/negative restrictions that are ANDed/ORed together, they are a flat set of positive restrictions that are ORed together. They are stored as strings in a table called selrestrs, just like synrestrs, which makes them a lot more queryable now as well. This change required some changes to the VerbNet data, because we needed to consolidate any ANDed clauses into single selrestrs, as well as convert any negative selrestrs into positive ones. The changes made are detailed on the wiki. Preposition choices are now encoded as comma-separated lists instead of using JSON. This change, along with the selrestrs one, allows us to remove verbly's dependency on nlohmann::json.
* Renamed object join fields to prevent conflicts with class namesKelly Rauchenberger2017-02-0315-52/+52
| | | | This was not a problem with clang but it caused compilation errors with gcc.
* Fixed reference to local address bug with gccKelly Rauchenberger2017-02-031-3/+6
| | | | Using the ternary operator appeared to cause a reference to local address bug with field::getConditionField. Replacing the ternary operator with an if statement fixes the problem. This problem did not occur with clang.
* Fixed statement generation re negative joins without a through tableKelly Rauchenberger2017-02-031-1/+1
| | | | | | | | Previously, a simple negative join directly to an object rather than through another table would error because the statement generator would attempt to instantiate a CTE on the field's through table, which is undefined. Now, the proper table is used regardless of whether a through table is defined.
* Added enum inequality matches to fieldKelly Rauchenberger2017-02-033-1/+25
|
* Restructured verb frame schema to be more queryableKelly Rauchenberger2017-01-2835-649/+885
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Groups are much less significant now, and they no longer have a database table, nor are they considered a top level object anymore. Instead of containing their own role data, that data is folded into the frames so that it's easier to query; as a result, each group has its own copy of the frames that it contains. Additionally, parts are considered top level objects now, and you can query for frames based on attributes of their indexed parts. Synrestrs are also contained in their own table now, so that parts can be filtered against their synrestrs; they are however not considered top level objects. Created a new type of field, the "join where" or "condition join" field, which is a normal join field that has a built in condition on a specified field. This is used to allow creating multiple distinct join fields from one object to another. This is required for the lemma::form and frame::part joins, because filters for forms of separate inflections should not be coalesced; similarly, filters on differently indexed frame parts should not be coalesced. Queries can now be ordered, ascending or descending, by a field, in addition to randomly as before. This is necessary for accessing the parts of a verb frame in the correct order, but may be useful to an end user as well. Fixed a bug with statement generation in that condition groups were not being surrounded in parentheses, which made mixing OR groups and AND groups generate inaccurate statements. This has been fixed; additionally, parentheses are not placed around the top level condition, and nested condition groups with the same logic type are coalesced, to make query strings as easy to read as possible. Also simplified the form::lemma field; it no longer conditions on the inflection of the form like the lemma::form field does. Also added a debug flag to statement::getQueryString that makes it return a query string with all of the bindings filled in, for debug use only.
* Removed some debug outputKelly Rauchenberger2017-01-243-4/+0
|
* Whitespace changesKelly Rauchenberger2017-01-2432-801/+801
|
* Added word::getGroupKelly Rauchenberger2017-01-242-3/+35
|
* Fixed behavior of normalizing grouped hierarchal joinsKelly Rauchenberger2017-01-242-57/+82
| | | | | | | | Previously, we did not merge grouped hierarchal joins; however, because of the way statements are compiled, we do need to merge OR-d positive hierarchal joins, and ANDed negative hierarchal joins. Also made some whitespace changes.
* Fixed notion::prepositionGroup fieldKelly Rauchenberger2017-01-231-13/+13
|
* Added ability to filter on existence of inflectionKelly Rauchenberger2017-01-234-3/+17
|
* Added form::startsWithVowelSound convenience methodKelly Rauchenberger2017-01-232-51/+77
|
* Rewrote tokensKelly Rauchenberger2017-01-234-650/+405
|
* Added verb frame parsingKelly Rauchenberger2017-01-239-260/+643
|
* Added filter compactingKelly Rauchenberger2017-01-233-42/+76
| | | | | Before statement compilation, empty filters are removed from group filters, and childless group filters become empty filters.
* Fixed normalization of negative join filtersKelly Rauchenberger2017-01-231-183/+192
| | | | | | | | | | | | Previously, negative join filters were folded in with positive joins by AND/ORing them together and negating the negative joins. Checking for the existence of something that doesn't match a condition is different from checking for the non-existence of something that does match a condition, so now normalization considers positive and negative join filters to be distinct classes of filters and does not fold them together. Also made some whitespace changes.
* Fixed nullity/non-nullity filters on join fieldsKelly Rauchenberger2017-01-231-2/+12
|
* Whitespace changesKelly Rauchenberger2017-01-231-19/+19
|
* Fixed generator ignoring multiple inflection variantsKelly Rauchenberger2017-01-221-257/+268
| | | | | | Previously, the generator would recognize at most one form per inflection per lemma; now, the generator adds all variants in AGID to the database.
* Removed underscores in two-word literal prepositions in verb framesKelly Rauchenberger2017-01-221-2/+11
|
* Fixed statement generation involving negative subqueriesKelly Rauchenberger2017-01-212-57/+234
| | | | | | | | | | Previously, we generated negative subqueries by integrating them into the main statement normally, and then making the connecting join be a LEFT JOIN instead of an INNER JOIN, and by adding a condition that the join column be NULL. The problem with this is that if the top table of the subquery joins against any other table (which join throughs always do), then no rows will be returned. This was solved by putting the subquery into a CTE and then LEFT JOINing as before with the CTE.
generated by cgit-pink 1.4.1 (git 2.36.1) at 2024-11-21 21:05:14 +0000