This document contains detailed information about declarative quality management in the FrameNet data realease R1.3. We use Consistent Document Engineering Toolkit (CDET) - a piece of general-purpose quality management software. Here, we describe all consistency rules applied to the Frame Database, the Lexical Database, and FrameNet documentation, and say how R1.3 conforms to these rules. Details about our general quality assurance architecture including both declarative and imperative measures can be found in:
J. Scheffczyk and M. Ellsworth, Improving the Quality of FrameNet, In Proc. of the LREC'06 Wkshp. on Quality assurance and quality measurement for language and speech resources, Genoa, Italy, pp 8 - 13.
Each of the rules are listed below by data-category, with a semi-informative name, the type of impact the violation has, and an importance level (which the FrameNet team uses to prioritize repairs). Within the data-categories, rules are ordered by their importance. Along with a prose description of the rules, we give examples about possible or real violations.
Before proceeding to the detailed list, it is useful to be more precise about the "types of impact" for violations mentioned above and the data-categories. In order of seriousness, the types of impact referred to below include:
The consistency rules are sorted by the categories of data examined:
frames_have_LUs_if_not_non-lexical
impact: Definition-violating
importance: 200
For each Frame we require at least one LU, except for those Frames that have a semantic type "Non-Lexical".
Abstract Frames without LUs do not have a corresponding word in a language, we mark these Frames with the semantic type "Non-Lexical".
A violation to this rule would be a frame without LUs that is not marked as "Non-Lexical".
frames_without_LUs_are_non-lexical
impact: Definition-violating
importance: 200
For each Frame we require at least one LU, except for those Frames that have a semantic type "Non-Lexical".
Abstract Frames without LUs do not have a corresponding word in a language, we mark these Frames with the semantic type "Non-Lexical".
A violation to this rule would be a frame without LUs that is not marked as "Non-Lexical".
frames_without_LUs_are_non-lexical
impact: Definition-violating
importance: 200
If a frame has any LUs then its semantic type should not be "Non-Lexical"
This corresponds rule frames_have_LUs_if_not_non-lexical above in the opposite direction.
coreset_FEs_are_core_and_member
impact: Definition-violating
importance: 200
For each FrameElement that is part of a CoreSet we require that this FrameElement is actually core (or core unexpressed) and a member of the Frame associated with the CoreSet relation.
By definition, a CoreSet for a frame f contains core FEs of f only.
This rule might be violated by CoreSets that contain an FE that is not core (which might happen by changing its coreness type) or that reference an FE of another frame (which is pretty unlikely and would require that an FE would change its frame).
nonempty_frames
impact: Definition-violating
importance: 200
Each Frame has at least one frame element.
By definition, frames without FEs are non-sensical because they would lack any fillers.
A violation to this rule would be a frame without any FEs.
frames_have_core_FE
impact: Definition-violating
importance: 200
Each Frame has at least one core (or core-unexpressed) frame element.
This rule is more demanding that rule nonempty_frames above:
It requires that at least one FE is core.
A violation to this rule would be a frame without any core (or core-unexpressed) FEs.
no_childRef_for_CoreSet_Excludes_Requires
impact: Definition-violating
importance: 170
There should be no source reference for the Frame relations CoreSet, Excludes, and Requires.
Those relations are unary in the way that they say that there are FEs in this Frame that belong to the CoreSet, exclude each other, or require one another.
So to speak, they are merely holders for FE relations and do not connect different Frames.
A violation would be a CoreSet Frame relation relation with a source reference to some Frame.
valid_childRef_for_all_but_CoreSet_Excludes_Requires
impact: Definition-violating
importance: 170
Every Frame relation (except CoreSet, Excludes, Requires, ReFraming) should have a valid (i.e. non-null and existent) source reference.
This rule corresponds to rule no_childRef_for_CoreSet_Excludes_Requires above.
It requires valid sources for all other Frame relations, which do connect different Frames.
valid_parentRef_for_all_frame_relations
impact: Definition-violating
importance: 170
Every Frame relation should have a valid (i.e. non-null and existent) target reference.
This is just a simple data integrity rule that applies to all Frame relations.
Unary Frame relations (CoreSet, Excludes, Requires, ReFraming) connect to their Frame via the relation target (not its source).
A violation to this rule would be a Frame relation without target reference or with a target reference that does not exist (which should not happen due to database strict constraints).
each_frame_is_connected
impact: Use-impacting
importance: 160
Each Frame is connected to at least some other frame by a Frame-to-Frame relation, i.e., there are no lonely Frames.
We want FrameNet to be a fully connected graph such that every frame is reachable from every other frame.
Notice that the rule is actually more relaxed in that it would allow "islands" of multiple frames that are not connected to other "islands".
Notice further that we neglect SeeAlso relations here.
In the S-DAG we find Frames that have no connections at all, except for the SeeAlso relation.
only_one_relation_inherits_using_subframe_causative_inchoative_perspective_on
impact: Correlated with error
importance: 155
Between two Frames there should exist at most one relation of type Inheritance, Using, SubFrame, Inchoative of, and Causative of.
In general, we prohibit multiple relations between two Frames.
There are, however exceptions for cases like the Frames Assistance and Intentionally_act.
An Assistance event actually evokes two Intentionally_act events:
One for the agent who assists (Inheritance) and one for the agent who is assisted (Using).
A violation to this rule would be two frames connected via more than one frame relation.
no_cycles_for_Inheritance_Using_Inchoative_of_Causative_of
impact: Definition-violating
importance: 155
The Frame-to-Frame relations Inheritance, Using, Inchoative of, and Causative of should be acyclic.
Except for the Precedes and Perspective_on relations, all frame relations must not be cyclic and form real hierarchies.
Precedes relations may be cyclic, which then expresses recurring events.
Perspective_on relations are symmetric and, therefore, cyclic by nature.
A violation would be some frame A inheriting from frame B, using some frame C, which in turn is an Inchoative frame of A.
valid_childRef_for_all_FE_relations_fast
impact: Definition-violating
importance: 150
Every Frame Element relation should have a valid (i.e. non-null and existent) source reference.
This is a simple integrity constraint similar to the integrity rules for Frame relations.
Notice that all FE relations are binary by definition.
valid_targetRef_for_all_FE_relations_fast
impact: Definition-violating
importance: 150
Every Frame Element relation should have a valid (i.e. non-null and existent) target reference.
This is a simple integrity constraint similar to the integrity rules for Frame relations.
Notice that all FE relations are binary by definition.
frame_elements_same_name_mapped_for_inheritance_using_causativeof_inchaoativeof_subframe_perspective_on
impact: Correlated with error
importance: 135
If two Frames are related (by inheritance, using, causative of, inchoative of, subframe perspective of relation) then those Frame Elements having the same names should be mapped.
Although there is no formal reason for this rule, we require it for the mere sake of good documentation.
Formally, FE names have no inherent semantics; FE names purely aid human users in understanding FrameNet.
A violation to this rule would be a frame A linked to a frame B, where both frames have some FE F with the same name that are not linked.
Most of the rules below model what is commonly called proper inheritance: A daughter frame should be more specific than each of its ancestor frames.
core_FEs_are_inherited_if_not_in_coreset_and_not_in_excludes
impact: Definition-violating
importance: 145
All core Frame Elements of a Frame are inherited if they are not part of a coreset or an excludes set of this Frame.
Core FEs are the required fillers of a frame.
FEs without any frame internal FE relations (i.e., CoreSet, Excludes) are conjunctively connected.
Therefore, the daughter frame should have at least all the required fillers of its ancestor frame in order to be more specific.
Notice that below we define more specific rules that take care of coreset and excluding FEs.
A violation to this rule would be a daughter frame that lacks a core FE of an ancestor frame.
coreset_FEs_are_inherited
impact: Definition-violating
importance: 145
At least one Frame Element of each coreset of a Frame is inherited.
FEs in a CoreSet are disjunctively connected.
A daughter frame is more specific if some CoreSet FEs of the parent frame are not inherited, i.e., there are fewer options to choose from.
Therefore, it is sufficient if only one of the coreset FEs is inherited.
A violation to this rule would be a CoreSet in a parent frame no FE of which is inherited to the daughter frame.
excluding_FEs_are_inherited_if_one_core
impact: Definition-violating
importance: 145
Of two mutually excluding FEs of a Frame either both are not core or at least one of them is core and inherited.
The rule follows the pattern of the rule coreset_FEs_are_inherited above.
Excluding FEs are in an "XOR" relationship and therefore, at least one of them should be inherited if core.
Notice that we permit that if both FEs in the excluding relationship are core then one of them might not be inherited.
A violation to this rule would be an uninherited core FE taking part in an excluding relationship with a non-core FE.
inheritance_mapped_FEs_extrathematic
impact: Definition-violating
importance: 140
Inherited Frame Elements should not change the coreness type from non-extrathematic to extrathematic.
This rule is comprised by inheritance_mapped_FEs_same_type but it is much more important.
inheritance_FE_property_coreset_inherited_reverse
impact: Definition-violating
importance: 140
The coreset property of frame elements propagated up the inheritance relation, i.e.: Each frame element in a coreset must have at least one corresponding frame element in the parent frame. All these parent frame elements must be part of one coreset of the parent frame.
In other words, the daughter frame must not introduce new members to CoreSets.
This rule is necessary from the logic point of view, saying that the child frame is more specific than the parent frame (you must not introduce new frame elements into a core set because this would open the semantic space of the child frame instead of narrowing it).
That means that the restrictions of the child frame must imply the restricions of the parent frame.
So for two frame elements a and b the rule says that if we have "a or b" in the child frame we should not have "a and b" in the parent frame.
As can be seen from the number of violations, this rule is not well
followed in the FrameNet data. After careful review and repair of the
violations which can be addressed, all of the violations remaining
show exactly the same character: the CoreSet in the daughter frame is
made up of FEs with a metonymic relation, from which the inherited FE
is always inferrable. For example, in the Duplication frame,
inheriting from Intentionally_create, the FE Copy, corresponding directly
to the Created_entity FE of the Intentionally_create frame, can be
used but it is also possible to use the FE Original. For a particular
Duplication event, it is always possible to recover the referent of
the Copy from a mention of the Original.
The current FrameNet position is that such cases still constitute
valid inheritance, but to license cases such as these while excluding
other relations which do not constitute valid inheritance, we will
have to make the metonymic connections between the FEs explicit and
exclude such cases of explicit metonymy from this rule. No such
relations were added before this data release, so the exceptions
remain unaddressed.
inheritance_FE_property_requires_inherited_reverse
impact: Definition-violating
importance: 140
The requires property of frame elements is propagated up the inheritance relation, i.e.: For each frame element f1 requiring another FE f2 of the child frame we determine the corresponding frame elements f1' and f2' in the parent frame. If both exist then they also should require each other.
This rule is necessary from the logic point of view, saying that the child frame is more specific than the parent frame.
That means that the restrictions of the child frame must imply the restricions of the parent frame.
Frame elements that require each other are logically connected via "implies".
So for two frame elements a and b the rule says that if we have "a implies b" in the child frame we should not have "a and b" in the parent frame.
inheritance_FE_property_excludes_inherited_reverse
impact: Definition-violating
importance: 140
The excludes property of frame elements propageted up the inheritance relation, i.e.: For each frame element f1 excluding another FE f2 of the child frame we determine the corresponding frame elements f1' and f2' in the parent frame. If both exist then they also should exclude each other.
This rule is necessary from the logic point of view, saying that the child frame is more specific than the parent frame.
That means that the restrictions of the child frame must imply the restricions of the parent frame.
Frame elements that exclude each other are logically connected via "xor".
So for two frame elements a and b the rule says that if we have "a xor b" in the child frame we should not have "a and b" in the parent frame.
inheritance_FE_property_requires_inherited
impact: Correlated with error
importance: 138
The requires property of frame elements is inherited, i.e.: For each frame element f1 requiring another FE f2 of the parent frame we determine the corresponding frame elements f1' and f2' in the child frame. If both exist then they also should require each other or there is no frame element relation between them (which means that f1' and f2' are both(!) necessary).
This rule is motivated by the linguistic point of view but not really necessary from the logic point of view requireing proper inheritance (i.e., that the restrictions of the child frame should imply the restrictions of the parent frame).
inheritance_FE_property_excludes_inherited
impact: Correlated with error
importance: 135
The excludes property of frame elements is inherited, i.e.: For each frame element f1 excluding another FE f2 of the parent frame we determine the corresponding frame elements f1' and f2' in the child frame. If both exist then they also should exclude each other. This rule is nice to have from the linguistic point of view but not really necessary from the logic point of view (i.e., that the restrictions of the child frame should imply the restrictions of the parent frame).
inheritance_FE_property_coreset_inherited
impact: Correlated with error nbsp;
importance: 130
The coreset property of frame elements is inherited, i.e.: For each frame element in a coreset of the parent frame we determine the corresponding frame elements in the child frame. If there are at least 2 then these must be part of a coreset of the child frame.
This rule is motivated by the linguistic point of view but not really necessary from the logic point of view (i.e., that the restrictions of the child frame should imply the restrictions of the parent frame).
inheritance_mapped_FEs_same_type
impact: Nice to know
importance: 130
The Frame Elements that are mapped to each other as part of an inheritance relation should have the same Coreness type.
semantictypes_of_mapped_FEs_are_inherited_inheritance_relation
impact: Definition-violating
importance: 130
All semantic types of an FE are inherited if it is mapped to another FE as part of an inheritance relation.
This rule (as well as the other rules involving semantic types in this list) requires coherence between the FE inheritance hierarchy and the semantic type hierarchy.
In fact, a more specific FE should have the same or a more specific
semantic type. Although the FrameNet data contain many violations of
this rule, most are cases where the parent FE has a particular
semantic type, and the daughter FE has none. In all such cases, the
semantic type of the parent should be imposed on the daughter FE.
There are, however, cases in which both parent and daughter FEs have
explicit semantic types. In some cases, both types legitimately
apply, while in others, the semantic type of the daughter is incorrect
and should be replaced by the parent semantic type. Unfortunately,
this complex work was not completed for this data release.
inheritance_all_frame_STs_are_inherited
impact: Definition-violating
importance: 130
All semantic types of a parent frame (except the "Non ..." types) are inherited to all child frames.
This rule corresponds to rule semantictypes_of_mapped_FEs_are_inherited_inheritance_relation but concerns the semantic types of frames.
Notice that abstract frames (indicated by semantic types Non-Lexical_Frame and Non-perspectivalized_Frame) do not need to inherit their types.
Usually, these abstract frames reside quite high in the inheritance hierarchy and have several concrete frames (i.e., those that have corresponding words and LUs) below.
inheritance_mapped_FEs_same_name
impact: Nice to know
importance: 0
The Frame Elements that are mapped to each other as part of an inheritance relation should have the same names.
This is not really a formal rule, but good to know for AI applications.
Therefore, its priority is set to 0.
each_frame_is_connected_via_inheritance
impact: Use-impacting
importance: 0
Each Frame is in the inheritance hierarchy.
This rule corresponds to rule each_frame_is_connected but is limited to the inheritance relation.
It does not have a formal impact since from the linguistic point of view it is not necessary that each frame takes part in the inheritance hierarchy.
For maintainance purposes it is often nice to know which frames do not take part in the inheritance hierarchy.
inchoative-of_all_core_FEs_mapped
impact: Correlated with error
importance: 125
For an Inchoative_of relation, each core (or core-unexpressed) FE of the parent Frame should be mapped to an FE of the child Frame.
causative-of_all_core_FEs_mapped_except_Agent_Cause
impact: Correlated with error
importance: 120
For a Causative_of relation, each core (or core-unexpressed) FE (except perhaps Agent or Cause) of the parent Frame should be mapped to an FE of the child Frame.
perspective_on_one_core_FEs_is_mapped
impact: Definition-violating
importance: 115
For a Perspective_on relation, one core (or core-unexpressed) FE of the parent Frame should be mapped to a FE of the child Frame.
subframe_one_core_FEs_mapped
impact: Definition-violating
importance: 115
For a Subframe relation, one core (or core-unexpressed) FE of the parent Frame should be mapped to a FE of the child Frame.
subframe_no_other_relations
impact: Definition-violating
importance: 115
If Frames A and B point to a common Frame C via Subframe then A and B should not be connected at all except by the precedence relation. (This is formalized via an exception.)
using_one_core_FEs_is_mapped
impact: Definition-violating
importance: 115
For a Using relation, one core (or core-unexpressed) FE of the parent Frame should be mapped to a FE of the child Frame.
subframe_precedes_same_subframe
impact: Definition-violating
importance: 114
If Frames A and B are connected via a precedes relation then they must be subframes of a common Frame C.
using_mapped_FEs_same_type
impact: Nice to know
importance: 110
The Frame Elements that are mapped to each other as part of a Using relation should have the same Coreness type. This is in no way a strict requirement, although if two frames were connected only via an extra-thematic FE, this would be a true violation. (The data were not checked for this more stringent requirement.)
semantictypes_of_mapped_FEs_are_inherited_using_relation
impact: Definition-violating
importance: 110
All semantic types of an FE are inherited if it is mapped to another FE as part of an using relation.
perspective_on_mapped_FEs_same_type
impact: Nice to know
importance: 110
The Frame Elements that are mapped to each other as part of a Perspective_on relation should have the same Coreness type.
using_all_frame_STs_are_inherited
impact: Nice to know
importance: 110
All semantic types of a parent frame are inherited to all child frames that use the parent Frame.
perspective_on_all_frame_STs_are_inherited
impact: Nice to know
importance: 110
All semantic types of a parent frame are inherited to all child frames that use the parent Frame.
semantictypes_of_mapped_FEs_are_inherited_perspective_on_relation
impact: Definition-violating
importance: 110
All semantic types of an FE are inherited if it is mapped to another FE as part of an perspective_on relation.
seealso_one_representative
impact: Definition-violating
importance: 100
If Frames A and B point to a common Frame C via Seealso then A and B should not be connected by Seealso.
seealso_target_mentions_source
impact: Definition-violating
importance: 30
If a Frame A points to B via Seealso then the definition of B should mention Frame A.
semantictypes_of_LUs_correspond_to_STs_of_frames
impact: Definition-violating
importance: 70
For each Frame we require that all of its semantic types are respected by all its LUs
lemmas_multiple_lexentries_exactly_one_head_and_equal_part_of_spch
impact: Correlated with error
importance: 65
For each Lemma connected to a LU we require that if it has multiple LexemeEntries then it exactly one of them is the head word and the part of speech of its associated Lexeme is the same as the part of speech of the Lemma.
lemmas_one_lexentry_same_POS
impact: Correlated with error
importance: 65
For each Lemma connected to a LU we require that if it has only one LexemeEntry then the part of speech of the associated Lexeme is the same as the part of speech of the Lemma.
lexUnit_description_proper
impact: Use-impacting
importance: 60
We require that the descriptions of LUs start either with FN: or COD:. This certainly excludes empty definitions, but also definitions that come from non-approved or non-indicated sources. While FrameNet has an agreement to use COD definitions and may make its own, other definitions would require a separate agreement.
frames_fes_name_abbrev_unique
impact: Use-impacting
importance: 49
Within each Frame the Names and Abbreviations of each Frame Element are unique. Especially in the case of FE Names, this is necessary for referencing them in the XML definitions.
frames_fe_definition_FEs_exist
impact: Use-impacting
importance: 48
Within the definition of a Frame Element belonging to a Frame F, for each referenced Frame Element we find a corresponding Frame Element (equal name or abbreviation) bound to the Frame F. Exceptions usually are due to FE names having been changed but not updated in the description of the frame's FEs.
frames_definition_FEs_exist
impact: Use-impacting
importance: 48
Within each Frame's definition for each referenced Frame Element we find a corresponding Frame Element (equal name or abbreviation) bound to this Frame.
frames_core_FEs_have_example_at_all
impact: Use-impacting
importance: 40
Each core (or core-unexpressed) FE of a (not non-lexical) Frame has examples (not necessarily marking this Frame Element).
frames_have_example_at_all
impact: Use-impacting
importance: 40
Each (not non-lexical) Frame has at least one example in the Frame definition.
frames_core_FEs_accounted_in_frame_definition
impact: Use-impacting
importance: 35
Each core (and core-unexpressed) FE of a Frame is accounted for in the Frame's definition. Most exceptions involve Core FEs added later than the Frames's definition was written.
frames_core_FEs_have_example_in_framedef
impact: Use-impacting
importance: 30
Each core (or core-unexpressed) FE of a (not non-lexical) Frame we find an example in the Frame definition.
frames_core_FEs_have_example_in_fedef
impact: Use-impacting
importance: 30
For each core (or core-unexpressed) FE of a (not non-lexical) Frame we find an example in the FE definition.
frames_FE_definition_targets_have_same_frame
impact: Use-impacting
importance: 30
Each annotated target of each Frame Element example of a Frame actually evokes that Frame.
frames_definition_targets_have_same_frame
impact: Use-impacting
importance: 30
Each annotated target in a Frame example actually evokes that Frame.
frames_core_FEs_definition_exist
impact: Nice to know
importance: 25
Each core (or core-unexpressed) FE of a Frame has a definition, i.e., it is referenced in its own definition.
docu_docs_parsable
impact: Use-impacting
importance: 20
Each document of the documentation is parsable.
docu_ref_fes_exist_in_db
impact: Use-impacting
importance: 10
For each frame element referenced in the documentation we find a frame element in the database with the same name.
docu_ref_frames_exist_in_db
impact: Use-impacting
importance: 10
For each frame referenced in the documentation we find a frame in the database with the same name.
docu_ref_LUs_Lexemes_exist_in_db
impact: Correlated with error
importance: 5
For each LU or Lexeme marked as \ment{} in the documentation we find a similar LU or a similar Lexeme in the database. Comparison is not case sensitive, white spaces might be any character in the DB. Multi-word LUs Lexemes are taken into account. Words separated by any punctuation symbol are treated as extra LUs / Lexemes.
docu_fenames_annotated
impact: Correlated with error
importance: 0
For each sentence we require that if the sentence is about frame elements then there is no not annotated word equal to a frame element name in the database (i.e. all frame element names are properly annotated).
docu_framenames_annotated
impact: Correlated with error
importance: 0
For each sentence we require that if the sentence is about frames then there is no not annotated word equal to a frame name in the database (i.e. all frame names are properly annotated).