MID Validation
Implementation
MID Validation checks are implemented as bins in the Workflow tab. When the check passes, it should return 0 results. The bins drive the action of the MIDValidationReport algorithm which runs as a cron job each Saturday.
- MID_VALIDATION: Queries that can additionally be made into checklists (i.e., in format clusterId, conceptId OR conceptId OR conceptId1, conceptId2)
- MID_VALIDATION_OTHER: Queries that cannot be made into checklists (i.e., the query doesn't not return a conceptId). These are not intended to be made into bins, and will not run correctly as bins. However, they still drive the action of MIDValidationReport.
To test either type of query, implement as a bin, and then use "Test" in the Edit Bin screen.
To implement "exceptions" to the checks, add them to the query.
Parameters
- terminology - e.g. NCIMTH
- version - e.g. latest
Query Patterns
Finding atoms for a concept (also works for a code or a descriptor)
select a.* from atoms a, concepts_atoms b, concepts c where a.id = b.atoms_id and b.concepts_id = c.id ...;
Find semantic types for a concept
select sty.* from concepts a, concepts_semantic_type_components b, semantic_type_components sty where a.id = b.concepts_id and b.semanticTypes_id = sty.id ...;
Find AUIs for an atom
select b.alternateTerminologyIds aui from atoms a, AtomJpa_alternateTerminologyIds b where a.id = b.AtomJpa_id and alternateTerminologyIds_KEY = :terminology ...;
Find SRC_ATOM_ID for an atom
select a.alternateTerminologyIds srcAtomId from atoms a, AtomJpa_alternateTerminologyIds b where a.id = b.AtomJpa_id and alternateTerminologyIds_KEY = concat(:terminology,'-SRC') ...;
Find last release CUI for an atom
select a.conceptTerminologyIds srcAtomId from atoms a, AtomJpa_conceptTerminologyIds b where a.id = b.AtomJpa_id and conceptTerminologyIds_KEY = :terminology ...;
Find the CUI for a concept (note these are not fully assigned until release time)
select c.terminologyId cui from concepts c where terminology = :terminology ...;
Find the ATUI for a semantic type (note these are not fully assigned until release time)
select sty.terminologyId atui from semantic_type_components where terminology = :terminology...;
Find the RUI for C level relationships (note these are not fully assigned until release time)
select r.terminologyId rui from concept_relationships r where terminology = :terminology ...;
Find the RUI for a SNOMED SCUI-SCUI relationship
select r.alternateTerminologyIds rui from concept_relationships r, ConceptRelationshipJpa_alternateTerminologyIds b where r.id = b.ConceptRelationshipJpa_id and alternateTerminologyIds_KEY = :terminology ...;
Find the sab/tty termgroups
-- default precedence list select a.id, sab.terminologies, tty.termTypes from precedence_lists a, precedence_list_terminologies sab, precedence_list_term_types tty where sab.terminologies_ORDER = tty.termTypes_ORDER and a.id = sab.PrecedenceListJpa_id and a.id = tty.PrecedenceListJpa_id and a.terminology = :terminology; -- project precedence list select a.id, sab.terminologies, tty.termTypes from precedence_lists a, precedence_list_terminologies sab, precedence_list_term_types tty where sab.terminologies_ORDER = tty.termTypes_ORDER and a.id = sab.PrecedenceListJpa_id and a.id = tty.PrecedenceListJpa_id and a.id in (select precedenceList_id from projects where terminology = :terminology);
Find inverse of a relationship
select * from additional_relationship_types....
Existing Checks
(to derive queries from or for)
Approved C rel matching demotion - DONE
select cr.from_id conceptId1, cr.to_id conceptId2 from atom_relationships ar, concept_relationships cr, concepts_atoms ca1, concepts_atoms ca2 where ar.terminology = :terminology and cr.terminology = :terminology and ar.from_id = ca1.atoms_id and ar.to_id = ca2.atoms_id and cr.from_id = ca1.concepts_id and cr.to_id = ca2.concepts_id and ar.workflowStatus = 'DEMOTION' and cr.workflowStatus in ('READY_FOR_PUBLICATION','PUBLISHED');
Atoms with leading/trailing junk - DONE, OTHER
select c.id, a.id conceptId from concepts c, concepts_atoms ca, (select id from atoms where name like ' %' or name like '% ') a where c.terminology=:terminology and c.id = ca.concepts_id and ca.atoms_id = a.id;
Atom with blank language value - DONE, OTHER
select c.id conceptId, a.id from concepts c, concepts_atoms ca, (select id from atoms where language='') a where ca.atoms_id = a.id and ca.concepts_id = c.id and c.terminology = :terminology;
N/A - Attribute value with multiple hashcodes 0 Sat Jan 7 09:02:19 2017 - n/a hascodes are internally compute
N/A - C level atts with concept_id,sg_id out of sync 0 Sat Jan 7 09:02:52 2017 - n/a cannot happen
N/A - C level rels with concept_id,sg_id out of sync 0 Sat Jan 7 09:03:01 2017 - n/a cannot happen
Codes with leading/trailing spaces - DONE, OTHER
select codeId, id as atomId, terminology from atoms a where codeId like ' %' or codeId like '% '
N/A - Conflicting context relationships 0 Sat Jan 7 09:03:18 2017
N/A - Conflicting relationships 0* Sat Jan 7 09:03:35 2017
Current sources without releasable atoms - DONE, OTHER
select terminology,version from terminologies b where current = true and not exists (select * from atoms a where a.terminology = b.terminology and a.version=b.version and a.publishable = true);
N/A - Current ver attribute connected to old ver object
N/A - Current ver relationoship connected to old ver object
N/A: Current ver attribute connected to old ver cxt rels
Current ver attribute connected to old ver concept rels - DONE, OTHER - *** add other types of rels
-- concept_relationships select at.id as attribute_id, c.id as concept_id, c.terminology from concept_relationships c, concept_relationships_attributes c_at, attributes at where c.id = c_at.concept_relationships_id and at.id = c_at.attributes_id and (c.terminology,c.version) in (select terminology,version from terminologies where current = false) and (at.terminology,at.version) in (select terminology,version from terminologies where current = true) and at.publishable = true;
Current ver relationship connected to old ver atom - DONE, OTHER
select a_r.id, a_r.from_id, a.terminology, a_r.to_id from atoms a, atom_relationships a_r where a.id = a_r.from_id and (a.terminology,a.version) in (select terminology,version from terminologies where current = false) and (a_r.terminology,a_r.version) in (select terminology,version from terminologies where current = true)
N/A - Dangling XMAP (no XMAPFROM) 0 Sat Jan 7 09:05:46 2017
N/A - Dangling XMAP (no XMAPTO) 0 Sat Jan 7 09:05:54 2017
N/A - Dangling XMAPFROM 0 Sat Jan 7 09:05:59 2017
N/A - Dangling XMAPTO 0 Sat Jan 7 09:06:02 2017
Mapset without mappings - DONE, OTHER
select a.id, a.name from mapsets a where a.publishable = true and not exists (select * from mappings b where a.id = b.mapset_id)
Subset without members - DONE, OTHER
select a.id, a.name from atom_subsets a where a.publishable = true and not exists (select * from atom_subset_members b where a.id = b.subset_id) union all select a.id, a.name from concept_subsets a where a.publishable = true and not exists (select * from concept_subset_members b where a.id = b.subset_id);
Duplicate relationship type inverse - DONE, OTHER
select inverse_id from relationship_types group by inverse_id having count(*)>1;
Duplicate additional relationship type inverse - DONE, OTHER
select inverse_id from additional_relationship_types group by inverse_id having count(*)>1;
N/A - Hashcode with multiple attribute values 0 Sat Jan 7 09:06:16 2017
Illegal C level relationship (rn=RT?,LK,status=R) - DONE, OTHER
select a.id from concept_relationships a where terminology = :terminology and relationshipType not in ('XR','RO','RB','RN','BRO','BRB','BRN');
N/A - llegal C level rels (status=U or rel=LK) 0 Sat Jan 7 09:06:20 2017
Source relationship with status=N - DONE, OTHER
select a.id from concept_relationships a where terminology != :terminology and workflowStatus = 'NEEDS_REVIEW' union all select a.id from atom_relationships a where workflowStatus = 'NEEDS_REVIEW' union all select a.id from code_relationships a where workflowStatus = 'NEEDS_REVIEW' union all select a.id from descriptor_relationships a where workflowStatus = 'NEEDS_REVIEW';
?? Inconsistent hierarchical code 0* Sat Jan 7 09:06:30 2017
N/A - Incorrect text_total in stringtab 0 Sat Jan 7 09:06:36 2017
N/A - Inverted context relationship attributes 0 Sat Jan 7 09:06:43 2017
Multiple C-level relationships - DONE, OTHER
select r.to_id conceptId1, r.from_id conceptId2 from concept_relationships r where terminology = :terminology group by r.to_id, r.from_id having count(*)>1;
N/A - Non-UTF8 attributes rows 0 Sat Jan 7 09:07:44 2017
N/A - Non-UTF8 string_ui rows 0 Sat Jan 7 09:07:47 2017
N/A - Non-UTF8 stringtab rows 0 Sat Jan 7 09:07:55 2017
N/A - Preferred atoms without contexts 0 Sat Jan 7 09:08:14 2017
N/A - Releasable QA concepts 0 Sat Jan 7 09:08:14 2017
N/A - Releasable old version rels 0 Sat Jan 7 09:08:14 2017
N/A - Releasable old version foreign sources 0 Sat Jan 7 09:08:14 2017
N/A - S level atts with aui,sg_id out of sync 0 Sat Jan 7 09:09:31 2017
N/A - S level rels with aui,sg_id out of sync 0 Sat Jan 7 09:11:30 2017\
N/A - SUBSET_MEMBER atts without corresponding SB atom 0 Sat Jan 7 09:11:49 2017
Self-referential C level relationships 0 Sat Jan 7 09:11:51 2017 - DONE
select r.from_id conceptId from concept_relationships r where terminology = :terminology and from_id = to_id;
N/A - Strings with junk characters 0 Sat Jan 7 09:11:55 2017
N/A - Termgroup (termgroup_rank) does not match tty 0 Sat Jan 7 09:11:55 2017
N/A - Termgroup does not match tty 0 Sat Jan 7 09:11:57 2017
Unexpected source with suppress = E -1* Sat Jan 7 09:12:04 2017 - DONE, OTHER
select distinct a.terminology from atoms a, term_types b where a.termType = b.abbreviation and a.suppressible = true and b.suppressible = false and a.terminology not in ('CSP','CST','DXP','FMA','MTH','NCBI','RXNORM','UWDA');
Unexpected source with suppress = O 0* Sat Jan 7 09:12:09 2017 - DONE, OTHER
select distinct a.terminology from atoms a where a.obsolete = true and a.terminology not in ('CVX', 'FMA', 'GO','HCPCS','HPO','HL7V3.0','LNC','MDR','MGED','MVX','MTHSPL','NCI','NICHD','PDQ','RXNORM','VANDF');
Checks To Implement
Checking new src integrities.........Sat Jan 7 09:19:58 2017
Bad termgroup in versioned SRC concept - (Implemented on MEME-Edit 04/08/2022)
select distinct c1.id conceptId from atoms a1, concepts_atoms b1, concepts c1, atoms a2, concepts_atoms b2, concepts c2 where a1.id = b1.atoms_id and b1.concepts_id = c1.id and a2.id = b2.atoms_id and b2.concepts_id = c2.id and c1.terminology = :terminology and c2.terminology = :terminology and c1.id = c2.id and a1.publishable = true and a1.terminology = 'SRC' and a1.termType = 'VPT' and a2.publishable = true and a2.terminology = 'SRC' AND a2.termType not in ('VAB','VPT','VSY');
Code does not match SRC/RAB name - (Implemented on MEME-Edit 04/08/2022)
select c1.id conceptId from atoms a1, concepts_atoms b1, concepts c1, atoms a2, concepts_atoms b2, concepts c2 where a1.id = b1.atoms_id and b1.concepts_id = c1.id and a2.id = b2.atoms_id and b2.concepts_id = c2.id and c1.terminology = :terminology and c2.terminology = :terminology and c1.id = c2.id and a1.terminology = 'SRC' and a1.termType = 'RAB' and a2.terminology = 'SRC' and a2.code not like concat('V-',a1.codeId);
Code does not match SRC/VAB name 28 Sat Jan 7 09:20:00 2017 (OUTPUT CONTAINS COMPLICATIONS) SEE ATTACHMENT HERE
select c1.id conceptId from atoms a1, concepts_atoms b1, concepts c1, atoms a2, concepts_atoms b2, concepts c2 where a1.id = b1.atoms_id and b1.concepts_id = c1.id and a2.id = b2.atoms_id and b2.concepts_id = c2.id and c1.terminology = :terminology and c2.terminology = :terminology and c1.id = c2.id and a1.terminology = 'SRC' and a1.termType = 'VAB' and a2.terminology = 'SRC' and a2.code not like concat('V-',a1.codeId);
Concepts with duplicate atoms (Implemented on MEME-Edit 05/11/2022)
select concepts_id as conceptId from concepts_atoms group by conceptId, atoms_id having count(*) >1;
Non-code-based SRC version_of rels : where terminology='SRC' and additionalRelationshipType='version_of' and publishable=true) (Implemented on MEME-Edit 07/11/2022)
(atom_relationship and component_info_relationship versions not implemented)
SELECT from_id AS conceptId1, to_id AS conceptId2 FROM concept_relationships WHERE terminology = 'SRC'AND additionalRelationshipType = 'version_of'AND publishable = TRUE; SELECT c1.id as conceptId1, c2.id as conceptId2 FROM concepts c1, concepts c2, descriptor_relationships b, descriptors_atoms da1, descriptors_atoms da2, concepts_atoms ca1, concepts_atoms ca2 WHERE b.from_id = da1.descriptors_id AND b.to_id = da2.descriptors_id AND da1.atoms_id = ca1.atoms_id AND da2.atoms_id = ca2.atoms_id AND c1.id = ca1.concepts_id AND c2.id = ca2.concepts_id AND c1.terminology = 'NCIMTH' AND c2.terminology = 'NCIMTH' AND b.terminology='SRC' AND b.additionalRelationshipType='version_of' AND b.publishable=true; #ATOM RELATIONSHIPS ARE ALLOWED TO HAVE THIS CONDITION. QUERY NOT IMPLEMENTED SELECT ca1.concepts_id as conceptId1, ca2.concepts_id as conceptId2 FROM atom_relationships ar,concepts_atoms ca1, concepts_atoms ca2 WHERE ar.from_id = ca1.atoms_id AND ar.to_id = ca2.atoms_id AND ar.terminology='SRC' AND ar.additionalRelationshipType = 'version_of' AND ar.publishable=true;
N/A - Invalid type/qualifier for SRC translation_of rels 0 Sat Jan 7 09:21:04 2017
N/A - Current source without citation 0 Sat Jan 7 09:20:00 2017
Current sources without releasable SRC/VABs
select terminology from terminologies t where current = true and not exists (select * from atoms a where a.terminology = 'SRC' and a.publishable = true and a.termType = 'VAB' and a.codeId = concat('V-',t.terminology));
N/A - HCPT has CPT source family 0 Sat Jan 7 09:20:00 2017
Hierarchy treetop not in same concept as RHT 0 Sat Jan 7 09:20:48 2017
how to find treetop?
N/A - Missing sims_info data 0 Sat Jan 7 09:21:18 2017
N/A - Non-ENG root SRC without translation_of rel 0 Sat Jan 7 09:21:19 2017
Non-SRC version_of rels 0 Sat Jan 7 09:21:27 2017 (Implemented on MEME-Edit 08/04/2022)
select id from code_relationships where terminology!='SRC' and additionalRelationshipType='version_of' and publishable=true;
Obsolete source w/o bequeathal rel to root 1 Sat Jan 7 09:21:27 2017 (Implemented on MEME-Edit 08/04/2022)
SELECT c_a.concepts_id FROM atoms a, concepts_atoms c_a WHERE a.publishable = false AND a.terminology = 'SRC' AND a.termType = 'VAB' AND a.id = c_a.atoms_id AND SUBSTRING(a.codeId,3) IN (SELECT concat(terminology,'_',version) FROM terminologies t WHERE current = false) AND NOT EXISTS (SELECT * FROM concept_relationships r, concepts_atoms c_a2, atoms a2 WHERE a2.publishable=true AND a2.terminology='SRC' AND a2.termType='RAB' AND a2.id = c_a2.atoms_id AND c_a.concepts_id = r.from_id AND c_a2.concepts_id = r.to_id AND r.relationshipType='BRO' );
Obsolete sources with releasable SRC/VABs 16 Sat Jan 7 09:21:28 2017 (Implemented on MEME-Edit 08/04/2022)
SELECT c_a.concepts_id, SUBSTRING(a.codeId,3) obsolete_source FROM atoms a, concepts_atoms c_a WHERE a.publishable = true AND a.terminology = 'SRC' AND a.termType = 'VAB' AND a.id = c_a.atoms_id AND SUBSTRING(a.codeId,3) IN (SELECT concat(terminology,'_',version) FROM terminologies t WHERE current = false);
Official Name does not match SRC/VPT 0 Sat Jan 7 09:21:28 2017 (Implemented on MEME-Edit 08/03/2022)
--official name is terminology.preferredName SELECT c_a.concepts_id, a.name FROM atoms a, concepts_atoms c_a WHERE a.publishable=true AND a.terminology='SRC' AND a.termType='VPT' AND a.id = c_a.atoms_id AND NOT EXISTS (SELECT * from terminologies t where t.preferredName=a.name);
N/A - Releasable SRC/RHT not used as tree-top 0* Sat Jan 7 09:22:10 2017
Root SRC exists for terminology without any atoms 0* Sat Jan 7 09:22:20 2017 (Implemented on MEME-Edit 08/04/2022)
SELECT * FROM atoms a WHERE a.terminology='SRC' AND a.termType='RPT' AND a.publishable=true AND NOT EXISTS (SELECT * FROM atoms b WHERE b.terminology=SUBSTRING(a.codeId,3) AND b.publishable=true);
SRC concept no Intellectual Product STY 0 Sat Jan 7 09:22:25 2017 (Implemented on MEME-Edit 08/04/2022)
SELECT c.id FROM atoms a, concepts_atoms c_a, concepts c WHERE a.id = c_a.atoms_id AND c_a.concepts_id = c.id AND a.terminology='SRC' AND a.publishable=true AND NOT EXISTS (select sty.* from concepts_semantic_type_components cstc, semantic_type_components sty where c.id = cstc.concepts_id and cstc.semanticTypes_id = sty.id and sty.semanticType='Intellectual Product');
SRC rels must be owned by SRC 0 Sat Jan 7 09:22:40 2017
atom/code relationships connected to SRC atoms or codes must be owned by SRC.
SRC/RAB missing from root_terminologies 0 Sat Jan 7 09:22:40 2017
SRC/RHT without contexts 0 Sat Jan 7 09:23:06 2017
no atom_tree_positions or concept_tree_positions or code/descriptor tree positions
SRC/VAB missing from terminologies 31 Sat Jan 7 09:23:06 2017
Source family is not RSAB of current source 0 Sat Jan 7 09:23:06 2017
there is a source family value in root_terminologies that does not match the abbreviation of a root terminology that has a current terminology version
N/A - Source version not set 0 Sat Jan 7 09:23:06 2017
Source with contexts and bad SRC/RHT 0 Sat Jan 7 09:23:06 2017
Source without exactly one SRC/RAB 0 Sat Jan 7 09:23:06 2017
Source without exactly one SRC/RPT 0 Sat Jan 7 09:23:06 2017
Source without exactly one SRC/SSN 0 Sat Jan 7 09:23:06 2017
Source without exactly one SRC/VAB 27* Sat Jan 7 09:23:06 2017
Source without exactly one SRC/VPT 0* Sat Jan 7 09:23:06 2017
Source with blank or null language/character_set/source_family 0* Sat Jan 7 09:23:06 2017
Source without name/short name 1 Sat Jan 7 09:23:06 2017
N/A - Verify TTY order has not changed 0 Sat Jan 7 09:23:06 2017
Versioned SRC without version_of rel 1 Sat Jan 7 09:23:06 2017
N/A - XM atoms in NCI Meta -1* Sat Jan 7 09:23:12 2017
N/A - source_rank.stripped_source minus source_version 0 Sat Jan 7 09:23:12 2017
N/A - source_version minus SRC/RAB 0 Sat Jan 7 09:23:12 2017
N/A - source_version.current_name minus source_rank 0 Sat Jan 7 09:23:12 2017
N/A - source_version.source minus source_rank 0 Sat Jan 7 09:23:12 2017
N/A - ~ MRDOC.EXPL with more than 1 value 8* Sat Jan 7 09:23:12 2017
Unimplemented Checks
Unexpected source with suppressible atts 0* Sat Jan 7 09:12:43 2017 (Implemented on MEME-Edit 10/18/2022)
select distinct a.id conceptId from attributes a where suppressible = true and terminology not in ('VANDF','MTHSPL');
N/A - Unexpected source with suppressible cxt rels 0 Sat Jan 7 09:13:08 2017
Unexpected source with suppressible relationships 0 Sat Jan 7 09:13:14 2017 (Implemented on MEME-Edit 10/18/2022)
select a.from_id as conceptId from atom_relationships a where a.suppressible = true union select a.from_id from concept_relationships a where a.suppressible = true and a.terminology != 'MTH' union select a.from_id from code_relationships a where a.suppressible = true union select a.from_id from descriptor_relationships a where a.suppressible = true;
N/A - Unreleasable current version rels 2924 Sat Jan 7 09:13:16 2017
N/A - Unreleasable parent in current source 0 Sat Jan 7 09:13:32 2017 Update sources with only zero last release rank 0 Sat Jan 7 09:13:48 2017 (NOT DOING)
select terminology,version from atoms where (terminology,version) in (select terminology,version from terminologies where current = false) group by terminology having max(lastPublishedRank) = '0';
Updated source now with CODE=NOCODE 0 Sat Jan 7 09:14:06 2017 (Implemented on MEME-Edit 11/09/2022)
select distinct terminology as conceptId from atoms where codeId = 'NOCODE' and terminology not in ('COSTAR','DXP','MCM','MTH','MTHCMSFRF','MTHMST','MTHSPL','NCIMTH'); #Original Query Below /* select distinct terminology,version from atoms where (terminology,version) in (select terminology,version from terminologies where current = false) and codeId = 'NOCODE'; */
N/A - Uppercase words 0 Sat Jan 7 09:14:07 2017
Validate CUI format 0 Sat Jan 7 09:14:15 2017 (Implemented on MEME-Edit 11/14/2022)
select id conceptId from concepts where terminology = :terminology and terminologyId != id and terminologyId not like 'C_______' and terminologyId not like 'CL_______';
N/A - ^Rels represented in both directions 91076* Sat Jan 7 09:15:35 2017
N/A - bad ATOM_NOTE attributes 0 Sat Jan 7 09:15:35 2017
N/A - bad CONCEPT_NOTE attributes 0 Sat Jan 7 09:15:35 2017
N/A - bad CONTEXT attributes 0 Sat Jan 7 09:15:47 2017
N/A - bad DEFINITION attributes 0 Sat Jan 7 09:19:03 2017
N/A - bad LEXICAL_TAG (TRD) attributes 0 Sat Jan 7 09:19:05 2017
N/A - bad LEXICAL_TAG (non-TRD) attributes 0 Sat Jan 7 09:19:06 2017
N/A - bad NON_HUMAN attributes 0 Sat Jan 7 09:19:06 2017
N/A - bad SEMANTIC_TYPE attributes 0 Sat Jan 7 09:19:18 2017
N/A - bad SFO/LFO relationships 0 Sat Jan 7 09:19:20 2017
N/A - bad SYNTACTIC_CATEGORY attributes 0 Sat Jan 7 09:19:20 2017
N/A - bad source level attributes 0 Sat Jan 7 09:19:32 2017
N/A - obsolete sources with releasable atoms 0 Sat Jan 7 09:19:41 2017
Incorrect publishable for concept level relationships
select a.from_id conceptId1, a.to_id conceptId2 from concept_relationships a where terminology = :terminology and ((relationshipType in ('XR','BRB','BRN','BRO') and publishable = true) OR (relationshipType not in ('XR','BRB','BRN','BRO') AND publishable = false));
N/A - ~ ATX mappings connected to bad CUIs 0 Sat Jan 7 09:19:43 2017
N/A - ~ Mixed status concepts 196 Sat Jan 7 09:19:47 2017
N/A - ~ Suppressible preferred name (level 0,9) 0 Sat Jan 7 09:19:58 2017
N/A - ~Required Editing Bins 0 Sat Jan 7 09:19:58 2017
Checking other referential integrities.........Sat Jan 7 09:23:12 2017
Metadata referential integrity (atom.tty, attribute.name, relationship.rel/rela, etc. atom.terminology/tty → precedence list - default or project, stycomponent->sty)
(Implemented on MEME-Edit 11/17/2022)
-- atom.tty select termType from atoms a where not exists (select * from term_types b where a.termType = b.abbreviation) and a.publishable = true; -- attribute.name select name from attributes a where not exists (select * from attribute_names b where a.name = b.abbreviation) and a.publishable = true; -- relationship.rel select relationshipType from concept_relationships a where not exists (select * from relationship_types b where a.relationshipType= b.abbreviation) and a.publishable = true union select relationshipType from atom_relationships a where not exists (select * from relationship_types b where a.relationshipType= b.abbreviation) and a.publishable = true union select relationshipType from code_relationships a where not exists (select * from relationship_types b where a.relationshipType= b.abbreviation) and a.publishable = true union select relationshipType from descriptor_relationships a where not exists (select * from relationship_types b where a.relationshipType= b.abbreviation) and a.publishable = true; -- relationship.rela -- same as above but "additionalRelationshipType" instead of "relationshipType" -- also include "atom_tree_positions", "concept_tree_positions", ... -- atom.terminology,tty select terminology, termType from atoms a where not exists (select * from precedence_lists b, precedence_list_terminologies c, precedence_list_termtypes d where b.id = c.PrecedenceListJpa_id and b.id = d.PrecedenceListJpa_id and b.terminology = 'NCIMTH' and a.terminology = c.terminologies and a.termType = d.termTypes and c.terminologies_ORDER = d.termTypes_ORDER) and a.publishable = true;
- terminology fields for all objects OR make a foreign key constraint from each of the tables to "terminologies.terminology" (Implemented on MEME-Edit 11/14/2022)
select terminology from ( select terminology from atoms where publishable = true union select terminology from codes where publishable = true union select terminology from descriptors where publishable = true union select terminology from concepts where publishable = true union select terminology from atom_relationships where publishable = true union select terminology from concept_relationships where publishable = true union select terminology from code_relationships where publishable = true union select terminology from descriptor_relationships where publishable = true union select terminology from atom_subset_members where publishable = true union select terminology from atom_subsets where publishable = true union select terminology from atom_transitive_rels where publishable = true union select terminology from atom_tree_positions where publishable = true union select terminology from attribute_identity union select terminology from attribute_names where publishable = true union select terminology from attributes where publishable = true union select terminology from code_transitive_rels where publishable = true union select terminology from code_tree_positions where publishable = true union select terminology from component_histories where publishable = true union select terminology from component_info_relationships where publishable = true union select terminology from concept_subset_members where publishable = true union select terminology from concept_subsets where publishable = true union select terminology from concept_transitive_rels where publishable = true union select terminology from concept_tree_positions where publishable = true union select terminology from deep_atom_relationships where publishable = true union select terminology from deep_concept_relationships where publishable = true union select terminology from definitions where publishable = true union select terminology from descriptor_tree_positions where publishable = true union select terminology from general_concept_axioms where publishable = true union select terminology from general_metadata_entries where publishable = true union select terminology from label_sets where publishable = true union select terminology from languages where publishable = true union select terminology from lexical_classes where publishable = true union select terminology from mappings where publishable = true union select terminology from mapsets where publishable = true union select terminology from relationship_types where publishable = true union select terminology from semantic_type_components where publishable = true union select terminology from semantic_types where publishable = true union select terminology from string_classes where publishable = true union select terminology from term_types where publishable = true ) a where not exists (select * from terminologies b where a.terminology = b.terminology);
Invalid language in atoms 0 Sat Jan 7 09:27:07 2017 (Implemented on MEME-Edit 11/14/2022)
select id from atoms where language != 'ENG';
Invalid semantic types (A) 0 Sat Jan 7 09:28:13 2017 (Implemented on MEME-Edit 11/14/2022)
select id,semanticType from semantic_type_components where semanticType not in (select expandedForm from semantic_types);
N/A - Max CL CUI should match value in max_tab 0 Sat Jan 7 09:29:08 2017 - n/a
Mismatched inverse_rel_attributes 0 Sat Jan 7 09:29:08 2017
additional_relationship_types.abbreviation - additional_relationship_types.inverse (Implemented on MEME-Edit 11/14/2022)
select abbreviation, inverse_id from additional_relationship_types where inverse_id is null union select b.abbreviation, b.id from additional_relationship_types a, additional_relationship_types b, additional_relationship_types c where a.inverse_id = b.id and b.inverse_id = c.id and a.id != c.id and a.publishable = true and c.publishable = true;
Mismatched inverse_relationships 0 Sat Jan 7 09:29:08 2017 (Implemented on MEME-Edit 11/14/2022)
select abbreviation, inverse_id from relationship_types where inverse_id is null union select b.abbreviation, b.id from relationship_types a, relationship_types b, relationship_types c where a.inverse_id = b.id and b.inverse_id = c.id and a.id != c.id and a.publishable = true and c.publishable = true;
N/A relationship_types.abbreviation - relationship_types.inverse
Missing ancestor node 0 Sat Jan 7 09:32:12 2017
(Brian is trying to optimize this one)
-- play around with "1~2~3~4" and try to get '4' and '1~2~3' select a.ancestorPath from atom_tree_positions a, atoms where a.node_id = atoms.id and not exists (select * from atom_tree_positions b where SUBSTRING_INDEX(a.ancestorPath, '~', -1) = atoms.terminologyId and a.ancestorPath like '%~%' and LEFT(a.ancestorPath, LENGTH(a.ancestorPath) - LENGTH(SUBSTRING_INDEX(a.ancestorPath, '~', -1)) - 1) = b.ancestorPath and a.terminology = b.terminology and a.additionalRelationshipType = b.additionalRelationshipType) -- repeat for concept_tree_positions, code_tree_positions, descriptor_tree_positions
N/A - handled above - Rows in classes with invalid sources 0 Sat Jan 7 09:35:10 2017
N/A - handled above Rows in classes with invalid termgroups 0 Sat Jan 7 09:35:18 2017
N/A - handled above - attributes with invalid source 0 Sat Jan 7 09:35:30 2017
N/A - handled above - relationships with invalid source 0 Sat Jan 7 09:35:30 2017
N/A - handled above - attribute wit invalid atn 0* Sat Jan 7 09:35:43 2017
N/A - handled above - atoms with invalid tty 0 Sat Jan 7 09:35:52 2017
N/A - handled above - relationsips with invalid rel/rela 0* Sat Jan 7 09:36:18 2017
N/A - handled above - atoms with invalid language 0 Sat Jan 7 09:36:18 2017
Checking source specific integrities.........Sat Jan 7 09:36:22 2017
TEST RESULTS
------------------------------------------------------ ----------
N/A - Check for GFG edits 0 Sat Jan 7 09:36:23 2017
N/A - Context TTYs check 0 Sat Jan 7 09:39:02 2017
Duplicate MTH CUI relationships 1166 Sat Aug 5 20:49:02 2017 (Seems like Multiple C-Level Relationships)
-- NOTE: we may already have a check like this, so it could be redundant -- check first select c1.id as conceptId1, c2.id as conceptId2 from concept_relationships cr, concepts c1, concepts c2 where cr.from_id = c1.id and cr.to_id = c2.id AND from_id<to_id and cr.terminology='NCIMTH' and c1.terminology='NCIMTH' and c2.terminology='NCIMTH' and cr.publishable = true and c1.publishable = true and c2.publishable = true GROUP BY cui1,cui2 HAVING COUNT(*)>1;
N/A - MSH foreign split MUIs 0 Sat Jan 7 09:39:02 2017
N/A - already checked - MTH self-referential relationships 3752 Sat Jan 7 09:39:04 2017
MTH-only concepts 0 Sat Jan 7 09:39:21 2017 (Implemented on MEME-Edit 12/02/2022)
-- NOTE: this may already exist, check first (OR see the fix for mth-only concepts, it likely has a query to find this) -- see query action process -> turn of MTH/PN-only concepts select c1.id as conceptId from concepts c, concepts_atoms ca, atoms a where c.id = ca.concepts_id and ca.atoms_id = a.id and c.publishable = true and a.publishable = true and (a.terminology = 'MTH' OR a.terminology = 'NCIMTH') and not exists (select * from concepts_atoms ca2, atoms a2 where ca2.atoms_id = a2.id and a2.publishable = true and ca1.concepts_id = ca2.concepts_id and a.terminology != 'MTH' and a.terminology != 'NCIMTH');
N/A - NOCODE MTH/{PT,MM} atoms 0 Sat Jan 7 09:39:21 2017
Duplicate RUI 0 Sat Jan 7 09:39:24 2017
Duplicate AUI
Duplicate ATUI
Duplicate SUI
Duplicate LUI
-- "duplicate" RUI select id from relationship_identity group by other fields,... having count(*)>1; -- duplicate ATUI ... same thing from attribute_identity ... -- same for AUI - atom_identity -- same for LUI - lexical_class_identity -- same for SUI - string_class_identity -- NOT for CUI
N/A - SIB rel with non null RELA (not UWDA) 0 Sat Jan 7 09:39:45 2017
N/A - Unreleasable MTH relationships 0 Sat Jan 7 09:39:47 2017
N/A - ~ MTH rels to split CUIs 43861 Sat Jan 7 09:40:34 2017
N/A - ~ concepts with multiple MTH/PN atoms 0 Sat Jan 7 09:40:36 2017
Checking string unique identifier integrity.........Sat Jan 7 09:40:36 2017
TEST RESULTS
------------------------------------------------------ ----------
use string_class_identity and lexical_class_identity
N/A - (norm_string,language)-LUI mismatch 0 Sat Jan 7 09:41:14 2017 (lexical_class_identity, atoms)
N/A - ENG strings with foreign language SUIs 0 Sat Jan 7 09:41:29 2017
N/A - Erroneous norm string 0 Sat Jan 7 09:41:40 2017
N/A - LUI-norm_string mismatch 0 Sat Jan 7 09:42:11 2017
Mismatches in string_ui,classes,atoms 0 Sat Jan 7 09:42:59 2017 (Implemented on MEME-Edit 11/18/2022)
-- NOTE: if the atoms stringClassId starts with an S you have to remove it select a.stringClassId, a.name atoms_name, s_c_i.name sci_name from atoms a, string_class_identity s_c_i where (substr(a.stringClassId,2))+0) = s_c_i.id and a.name != s_c_i.name;
Mismatches between atoms and string_class_identity, lexical_class_identity (Implemented on MEME-Edit 11/18/2022)
select id from atoms where terminology != 'SRC' and (name,(substr(stringClassId,2))+0) not in (select name,id from string_class_identity); select id from atoms where terminology != 'SRC' and (substr(lexicalClassId,2)+0) not in (select id from lexical_class_identity);
N/A - handled by above - Mismatches in string_ui,classes,normstr 0 Sat Jan 7 09:43:43 2017
Mismatches in string_ui,d_classes,d_atoms 0 Sat Jan 7 09:43:43 2017
Non-unique SUI in string_ui 0 Sat Jan 7 09:43:58 2017
Non-unique string,language in string_ui 0 Sat Jan 7 09:44:25 2017
Rows in classes minus string_ui (SUI) 0 Sat Jan 7 09:45:00 2017
Rows in classes minus string_ui(LUI) 0 Sat Jan 7 09:45:16 2017 (Already Implemented)
select id from atoms a where not exists (select * from string_class_identity b where b.id = (substr(a.stringClassId,2))+0) ); select id from atoms a where not exists (select * from lexical_class_identity b where b.id = (substr(a.lexicalClassId,2))+0) );
N/A - Rows in dead_classes minus string_ui (LUI) 0 Sat Jan 7 09:45:21 2017
N/A - Rows in dead_classes minus string_ui(SUI) 0 Sat Jan 7 09:45:50 2017
N/A - Rows in foreign_classes minus string_ui (LUI,lang) 0 Sat Jan 7 09:46:16 2017
N/A - Rows in foreign_classes minus string_ui (SUI,lang) 0 Sat Jan 7 09:46:49 2017
N/A - Rows w/ non-null LUI & null norm_string 0 Sat Jan 7 09:46:49 2017
N/A - Rows w/ null LUI & non-null norm_string 0 Sat Jan 7 09:46:49 2017
leading spaces 0 Sat Jan 7 09:46:57 2017 (Whitespace already implemented. Special chars implemented on MEME-Edit 12/07/2022)
-- leading, trailing or duplicate whitespace in names (if there are duplicate whitespace things, just remove that part) select id from atoms where (name like ' %' OR name like '% ' OR name like '% %') and publishable = true; -- check for \t, \r, or \n (look up ascii numbers for these 3 chars) SELECT * FROM atoms WHERE INSTR(name, char(10)) > 0 and publishable = true union SELECT * FROM atoms WHERE INSTR(name, char(09)) > 0 and publishable = true union SELECT * FROM atoms WHERE INSTR(name, char(13)) > 0 and publishable = true;
Checking unique identifier (ui) integrity.........Sat Jan 7 09:46:57 2017
TEST RESULTS
------------------------------------------------------ ----------
ATUI with bad source 0 Sat Jan 7 09:47:18 2017
TODO: edit the large query that checks terminology to include "atom_identity" and "relationship_identity"
n/a- AUI with bad root source 0 Sat Jan 7 09:47:21 2017
done - Bad ATUI (C rels - attributes_ui) 0 Sat Jan 7 09:48:35 2017
n/a Bad ATUI (S rels - attributes_ui) 0 Sat Jan 7 09:48:40 2017
Bad AUI (classes-atoms_ui) 0 Sat Jan 7 09:49:40 2017 (Implemented on MEME-Edit 12/07/2022)
select * from atomjpa_alternateterminologyids where alternateTerminologyIds_KEY = 'NCIMTH' AND not exists (select * from atom_identity b where b.id = (substr(a.alternateTerminologyIds,2)+0) );
Bad RUI (C rels - relationships_ui) 0 Sat Jan 7 09:50:14 2017 (Implemented on MEME-Edit 12/07/2022)
select * from atomrelationshipjpa_alternateterminologyids a where alternateTerminologyIds_KEY = 'NCIMTH' AND not exists (select * from relationship_identity b where b.id = (substr(a.alternateTerminologyIds,2)+0) ) UNION ALL select * from coderelationshipjpa_alternateterminologyids a where alternateTerminologyIds_KEY = 'NCIMTH' AND not exists (select * from relationship_identity b where b.id = (substr(a.alternateTerminologyIds,2)+0) ) UNION ALL select * from componentinforelationshipjpa_alternateterminologyids a where alternateTerminologyIds_KEY = 'NCIMTH' AND not exists (select * from relationship_identity b where b.id = (substr(a.alternateTerminologyIds,2)+0) ) UNION ALL select * from conceptrelationshipjpa_alternateterminologyids a where alternateTerminologyIds_KEY = 'NCIMTH' AND not exists (select * from relationship_identity b where b.id = (substr(a.alternateTerminologyIds,2)+0) ) UNION ALL select * from descriptorrelationshipjpa_alternateterminologyids a where alternateTerminologyIds_KEY = 'NCIMTH' AND not exists (select * from relationship_identity b where b.id = (substr(a.alternateTerminologyIds,2)+0) )
N/A - Bad RUI (NLM03 rels - relationships_ui) 0 Sat Jan 7 09:50:50 2017
N/A - Bad RUI (S rels - relationships_ui) 0 Sat Jan 7 09:54:43 2017
N/A - Bad RUI (context rels - relationships_ui) 0 Sat Jan 7 09:55:24 2017
N/A - Duplicate ATUI (attributes_ui) 0 Sat Jan 7 09:57:19 2017
N/A - Duplicate ATUI values (attributes_ui) 0 Sat Jan 7 09:57:53 2017
N/A - Duplicate AUI values 0 Sat Jan 7 09:58:30 2017
N/A - Duplicate RUI (inverse_relationships_ui) 0 Sat Jan 7 10:01:33 2017
N/A - Duplicate RUI (relationships_ui) 0 Sat Jan 7 10:03:47 2017
N/A - Duplicate RUI values (relationships_ui) 67 Sat Jan 7 10:04:26 2017
N/A - Duplicate atom_id's (C-dead_C) 0 Sat Jan 7 10:04:31 2017
N/A - Duplicate atom_id's (atoms-dead_atoms) 0 Sat Jan 7 10:04:40 2017
N/A - Duplicate atomic_action_id's 0 Sat Jan 7 10:04:41 2017
N/A - Duplicate attribute_id's (A-dead_A) 0 Sat Jan 7 10:04:55 2017
N/A - Duplicate concept_id's (CS-dead_CS) 0 Sat Jan 7 10:05:00 2017
Duplicate ids in source_id_map (C) 0 Sat Jan 7 10:05:11 2017 - alternate terminology ids
select alternateTerminologyIds from atomjpa_alternateterminologyids where alternateTerminologyIds_KEY = 'NCIMTH-SRC' group by alternateTerminologyIds having count(*)>1; -- if above has exceptions try select alternateTerminologyIds from atomjpa_alternateterminologyids where alternateTerminologyIds_KEY = 'NCIMTH-SRC' group by alternateTerminologyIds having count(distinct AtomJpa_id)>1;
Duplicate inverse RUI 0 Sat Jan 7 10:05:29 2017
-- ADD a unique index to relationship_identity inverseId select * from atomrelationshipjpa_alternatetermiologyids a, atomrelationshipjpa_alternatetermiologyids b, relationship_identity c where a.alternateTerminologyIds_KEY = 'NCIMTH' and b.alternateTerminologyIds_KEY = 'NCIMTH' and (substr(a.alternateTerminologyIds,2))+0) ) = c.id and (substr(b.alternateTerminologyIds,2))+0) ) = c.inverseId group by b.alternateTerminologyIds having count(distinct a.alternateTerminologyIds)>1; ... do this also for the other relationship alternate id tables.
N/A - Duplicate molecule_id's 0 Sat Jan 7 10:05:29 2017
N/A - Duplicate relationship_id's (CR-dead_CR) 0 Sat Jan 7 10:06:00 2017
N/A - Duplicate relationship_id's (R-dead_R) 0 Sat Jan 7 10:06:11 2017
N/A - Duplicate string_id's (stringtab-dead_stringtab) 0 Sat Jan 7 10:06:21 2017
N/A - Duplicate string_ids (A) 0 Sat Jan 7 10:07:01 2017
Inverse RUI check 237 Sat Jan 7 10:07:36 2017
-- ADD a unique index to relationship_identity inverseId select * from atomrelationshipjpa_alternatetermiologyids a where a.alternateTerminologyIds_KEY = 'NCIMTH' and not exists (select * from atomrelationshipjpa_alternatetermiologyids b, relationship_identity c where b.alternateTerminologyIds_KEY = 'NCIMTH' and (substr(a.alternateTerminologyIds,2))+0) ) = c.id and (substr(b.alternateTerminologyIds,2))+0) ) = c.inverseId); ... do this also for the other relationship alternate id tables.
N/A - Non-unique ATUI 0 Sat Jan 7 10:08:02 2017
N/A - Non-unique AUI 0 Sat Jan 7 10:08:38 2017
N/A - Non-unique AUI (foreign) 0 Sat Jan 7 10:08:38 2017
N/A - Non-unique RUI (CR) 0 Sat Jan 7 10:10:04 2017
N/A - Non-unique RUI (R union inverse R) 0 Sat Jan 7 10:11:38 2017
N/A - Non-unique RUI (R with relationship_group) 0 Sat Jan 7 10:14:06 2017
N/A - Non-unique RUI (R) 0 Sat Jan 7 10:14:12 2017
N/A - Non-unique RUI (R) - selfref 0 Sat Jan 7 10:14:14 2017
N/A - Overlapping relationship_id's (R-CR) 0 Sat Jan 7 10:14:24 2017
N/A - Overlapping relationship_id's (R-dead_CR) 0 Sat Jan 7 10:14:24 2017
N/A - Overlapping relationship_id's (dead_R-CR) 0 Sat Jan 7 10:14:26 2017
N/A - RUI with bad source 0 Sat Jan 7 10:14:49 2017
PICKUP HERE
Releasable atom with null AUI 0 Sat Jan 7 10:14:56 2017
atoms entry that doesn't have an alternateTerminologyIds entry for 'NCIMTH'
N/A - Releasable atom with null AUI (foreign) 0 Sat Jan 7 10:14:56 2017
Releasable attribute with null ATUI 0 Sat Jan 7 10:15:08 2017
Releasable rel with null RUI 0 Sat Jan 7 10:15:17 2017
Source rui without rel directionality 0 Sat Jan 7 10:15:26 2017
atom_id out of range (C-dead_C) 0 Sat Jan 7 10:15:27 2017
atom_id out of range (atoms-dead_atoms) 0 Sat Jan 7 10:15:27 2017
atomic_action_id out of range(atomic_actions) 0 Sat Jan 7 10:15:28 2017
attribute_id out of range (A-dead_A) 0 Sat Jan 7 10:15:40 2017
aui out of range 0 Sat Jan 7 10:15:48 2017
concept_id out of range (CS-dead_CS) 0 Sat Jan 7 10:15:49 2017
last_release_cui out of range (C) 0 Sat Jan 7 10:15:49 2017
molecule_id out of range (molecular_actions) 0 Sat Jan 7 10:15:49 2017
relationship_id out of range (CR-dead_CR) 0 Sat Jan 7 10:15:51 2017
relationship_id out of range (R-dead_R) 0 Sat Jan 7 10:15:51 2017
string_id out of range (A-dead_A) 0 Sat Jan 7 10:16:01 2017
string_id out of range ({dead_,}stringtab) 0 Sat Jan 7 10:16:03 2017
~ AUI split across CUI 0 Sat Jan 7 10:16:29 2017
~ Non-unique AUI (NLM02) 0 Sat Jan 7 10:16:29 2017
~ATUI with multiple attribute_ids 0 Sat Jan 7 10:17:42 2017
~AUI with multiple atom_ids 50 Sat Jan 7 10:17:55 2017
~RUI with multiple relationship_ids 0 Sat Jan 7 10:18:12 2017
Checking unique identifier (ui) referential integrity.........Sat Jan 7 10:18:12 2017
TEST RESULTS
------------------------------------------------------ ----------
C level attributes minus classes 0 Sat Jan 7 10:18:53 2017
C level relationships minus classes 0 Sat Jan 7 10:19:11 2017
Rows in CR minus classes 0 Sat Jan 7 10:19:37 2017
Rows in CS minus classes 0 Sat Jan 7 10:19:41 2017
Rows in atomic_actions (A) minus A 0 Sat Jan 7 10:20:42 2017
Rows in atomic_actions (C) minus classes 0 Sat Jan 7 10:20:51 2017
Rows in atomic_actions (CR) minus CR 0 Sat Jan 7 10:21:06 2017
Rows in atomic_actions (CS) minus CS 0 Sat Jan 7 10:21:12 2017
Rows in atomic_actions (R) minus R 0 Sat Jan 7 10:21:22 2017
Rows in atoms minus classes 0 Sat Jan 7 10:21:32 2017
Rows in attributes minus stringtab 0 Sat Jan 7 10:21:55 2017
Rows in classes minus CS 0 Sat Jan 7 10:22:00 2017
Rows in classes minus atoms 0 Sat Jan 7 10:22:09 2017
Rows in classes minus normstr 0 Sat Jan 7 10:22:46 2017
Rows in classes minus normwrd 0* Sat Jan 7 10:23:25 2017
Rows in classes minus source_id_map 66202 Sat Jan 7 10:24:02 2017
Rows in classes minus word_index 0* Sat Jan 7 10:24:41 2017
Rows in foreign_classes minus classes 0 Sat Jan 7 10:24:46 2017
Rows in normstr minus classes 0 Sat Jan 7 10:25:08 2017
Rows in normwrd minus classes 0 Sat Jan 7 10:25:21 2017
Rows in source_id_map minus classes 0 Sat Jan 7 10:25:32 2017
Rows in stringtab minus attributes 0 Sat Jan 7 10:25:55 2017
Rows in word_index minus classes 0 Sat Jan 7 10:26:10 2017
S level attributes minus classes 0 Sat Jan 7 10:27:31 2017
S level relationships minus classes 0 Sat Jan 7 10:28:25 2017
New Checks
ComponentInfoRels
- there are only certain combinations used in this table - see what they are and make a check to verify those are the only patterns
- e.g. source,from type, to type
Orphaned tracking records
select id from tracking_records tr left join checklists_tracking_records ctr on tr.id=ctr.trackingRecords_id left join worklists_tracking_records wtr on tr.id=wtr.trackingRecords_id left join workflow_bins_tracking_records wbtr on tr.id=wbtr.trackingRecords_id where ctr.trackingRecords_id is null and wtr.trackingRecords_id is null and wbtr.trackingRecords_id is null;
Verify SNOMED SRC concept structure (see snomed.csh recipe at the end)
Atoms without AUIs
SELECT id FROM atoms a WHERE NOT EXISTS (SELECT * from atom_identity a_i WHERE a.codeId = a_i.codeId AND a.conceptId = a_i.conceptId AND a.descriptorId = a_i.descriptorId AND a.stringClassId = a_i.stringClassId AND a.termType = a_i.termType AND a.terminology = a_i.terminology AND a.terminologyId = a_i.terminologyId ); SELECT id FROM atoms a WHERE NOT EXISTS (SELECT alternateTerminologyIds from AtomJpa_alternateTerminologyIds b WHERE a.id = b.AtomJpa_id and alternateTerminologyIds_KEY = :terminology);
Relationships without RUIs (non-NCIMTH)
SELECT id FROM atom_relationships r WHERE NOT EXISTS (SELECT alternateTerminologyIds FROM AtomRelationshipJpa_alternateTerminologyIds b WHERE r.id = b.AtomRelationshipJpa_id AND alternateTerminologyIds_KEY != :terminology); -- does not apply for concepts because NCIMTH SELECT id FROM code_relationships r WHERE NOT EXISTS (SELECT alternateTerminologyIds FROM CodeRelationshipJpa_alternateTerminologyIds b WHERE r.id = b.CodeRelationshipJpa_id AND alternateTerminologyIds_KEY != :terminology); SELECT id FROM descriptor_relationships r WHERE NOT EXISTS (SELECT alternateTerminologyIds FROM DescriptorRelationshipJpa_alternateTerminologyIds b WHERE r.id = b.DescriptorRelationshipJpa_id AND alternateTerminologyIds_KEY != :terminology); SELECT id FROM component_info_relationships r WHERE NOT EXISTS (SELECT alternateTerminologyIds FROM ComponentInfoRelationshipJpa_alternateTerminologyIds b WHERE r.id = b.ComponentInfoRelationshipJpa_id AND alternateTerminologyIds_KEY != :terminology);
Attributes without ATUIs
SELECT a.id FROM attributes a, atoms_attributes a_a, atoms b, AtomJpa_alternateTerminologyIds b_a WHERE a.id = a_a.attributes_id and b.id = a_a.atoms_id AND b.id = b_a.AtomJpa_id AND b_a.alternateTerminologyIds_KEY = :terminology AND (b_a.alternateTerminologyIds,b.terminology,'ATOM',a.name,a.terminology, a.terminologyId) NOT IN (SELECT componentId,componentTerminology, componentType, name, terminology, terminologyId FROM attribute_identity a_i WHERE a_i.componentType = 'ATOM'); SELECT a.id FROM attributes a, codes_attributes c_a, codes b WHERE a.id = c_a.attributes_id and b.id = c_a.codes_id AND (b.id, b.terminology,'CODE',a.name,a.terminology, a.terminologyId) NOT IN (SELECT componentId, componentTerminology, componentType, name, terminology, terminologyId FROM attribute_identity a_i WHERE a_i.componentType = 'CODE'); SELECT a.id FROM attributes a, concepts_attributes c_a, concepts b WHERE a.id = c_a.attributes_id and b.id = c_a.concepts_id AND (b.id,b.terminology,'CONCEPT',a.name,a.terminology, a.terminologyId) NOT IN (SELECT componentId, componentTerminology, componentType, name, terminology, terminologyId FROM attribute_identity a_i WHERE a_i.componentType = 'CONCEPT'); SELECT a.id FROM attributes a, descriptors_attributes d_a, concepts b WHERE a.id = d_a.attributes_id and b.id = d_a.descriptors_id AND (b.id,b.terminology,'DESCRIPTOR',a.name,a.terminology, a.terminologyId) NOT IN (SELECT componentId,componentTerminology, componentType, name, terminology, terminologyId FROM attribute_identity a_i WHERE a_i.componentType = 'DESCRIPTOR');
Relationships without inverses
-- Invalid inverseId in relationship_identity select id,inverseId from relationship_identity WHERE inverseId is null OR inverseId NOT IN (select id from relationship_identity); -- Invalid inverse_id values in relationship_identity SELECT id FROM relationship_identity WHERE NOT EXISTS (SELECT ri1.id, ri2.id FROM relationship_identity ri1, relationship_identity ri2 WHERE ri1.inverseId=ri2.id AND ri1.additionalRelationshipType= (SELECT art1.abbreviation FROM additional_relationship_types art1, additional_relationship_types art2 WHERE ri1.additionalRelationshipType = art1.abbreviation AND art1.terminology = :terminology AND art1.inverse_id = art2.id AND art2.terminology = :terminology AND art2.abbreviation = ri2.additionalRelationshipType ) AND ri1.terminology = ri2.terminology AND ri1.terminologyId=ri2.terminologyId AND ri1.fromId = ri2.toId AND ri1.fromTerminology = ri2.toTerminology AND ri1.fromType = ri2.toType AND ri1.relationshipType = (SELECT rt1.abbreviation FROM relationship_types rt1, relationship_types rt2 WHERE ri1.relationshipType = rt1.abbreviation AND rt1.terminology = :terminology AND rt1.inverse_id=rt2.id AND rt2.terminology = :terminology AND rt2.abbreviation = ri2.relationshipType ) AND ri1.terminology = ri2.terminology AND ri1.terminologyId=ri2.terminologyId AND ri1.toId = ri2.fromId AND ri1.toTerminology=ri2.fromTerminology AND ri1.toType = ri2.fromType );
DEFINITIONs not connected to core data
select id from definitions d WHERE publishable=true AND NOT EXISTS (SELECT * FROM (SELECT definitions_id FROM atoms_definitions UNION ALL SELECT definitions_id FROM concepts_definitions UNION ALL SELECT definitions_id FROM descriptors_definitions) a WHERE d.id = a.definitions_id);
Publishable old version content (atoms, attributes, atom classes, relationships, definitions, subset, mapset, subset member, mapping)
Publishable content (attribute, relationship, atom, subset member, mapping, definitions) connected to unpublishable object
atom "concept terminology ids" should exist for MTH$VER where VER is the current version of MTH.
Tree positions should exist only for "current" version terminologies.
mapset.toTerminology and toVersion must exist in terminologies