Versions Compared

Version Old Version 18 New Version 19
Changes made by

Rick Wood

Rick Wood

Saved on

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

...

  • nci_merge: Merged SCUI current version NCI atoms

    Code Block
    select distinct a.id conceptId 
from concepts a, concepts_atoms b, atoms c
where a.terminology = :terminology
  and a.id = b.concepts_id and b.atoms_id = c.id
  and c.terminology='NCI'
group by a.id having count(distinct c.conceptId)>1

  • nci_sub_split: Split SCUI current version NCI (or sub-source) atoms

    Code Block
    --Identify Sub-source atoms that aren't in the same concept as the NCI atom with the same conceptId
 
SELECT DISTINCT
    c.id conceptId1, c1.id conceptId2
FROM
    concepts c,
    concepts_atoms ca,
    atoms a,
    concepts c1,
    concepts_atoms ca1,
    atoms a1
WHERE
    c.terminology = :terminology
        AND c1.terminology = :terminology
        AND c.id = ca.concepts_id
        AND ca.atoms_id = a.id
        AND c1.id = ca1.concepts_id
        AND ca1.atoms_id = a1.id
        AND a.terminology = 'NCI'
        AND a1.terminology IN (SELECT 
            terminology
        FROM
            root_terminologies
        WHERE
            family = 'NCI' AND terminology != 'NCI')
        AND a.conceptId = a1.conceptId
        AND c.id != c1.id

  • sct_sepfnpt: SNOMED concept clusters where the FN and PT terms are separated

    Code Block
    SELECT DISTINCT
    c.id conceptId1, c1.id conceptId2
FROM
    concepts c,
    concepts_atoms ca,
    atoms a,
    concepts c1,
    concepts_atoms ca1,
    atoms a1
WHERE
    c.terminology = :terminology
        AND c1.terminology = :terminology
        AND a.terminology = 'SNOMEDCT_US'
        AND a1.terminology = 'SNOMEDCT_US'
        AND c.id = ca.concepts_id
        AND ca.atoms_id = a.id
        AND c1.id = ca1.concepts_id
        AND ca1.atoms_id = a1.id
        AND a.termType = 'FN'
        AND a1.termType = 'PT'
        AND a.conceptId = a1.conceptId
        AND c.id != c1.id

  • cdsty_coc: Find concepts with Clinical Drug STY and any other STY

    Code Block
    SELECT DISTINCT
    c.id conceptId
FROM
    concepts c,
    concepts_semantic_type_components cs
WHERE
    c.terminology = :terminology
        AND c.id = cs.concepts_id
        AND cs.concepts_id IN (SELECT 
            cs.concepts_id
        FROM
            concepts c,
            concepts_semantic_type_components cs,
            semantic_type_components s
        WHERE
            c.terminology = :terminology
                AND c.id = cs.concepts_id
                AND cs.semanticTypes_id = s.id
                AND s.semanticType = 'Clinical Drug')
GROUP BY cs.concepts_id
HAVING COUNT(cs.concepts_id) > 1

  • multsty:concepts with more than 3 STYs|

    Code Block
    SELECT DISTINCT
    cs.concepts_id conceptId
FROM
	concepts c,
    concepts_semantic_type_components cs,
    semantic_type_components s
WHERE
	c.terminology = :terminology
	AND c.id = cs.concepts_id
    AND cs.semanticTypes_id = s.id
GROUP BY c.id
HAVING COUNT(c.id) > 3

  • styisa:One STY is an ancestor of another in the STY isa hierarchy

    Code Block
    select t1.conceptId from 
(select c.id conceptId, st.id styId, st.treeNumber from concepts c, concepts_semantic_type_components cstc, (select stc.id, st.treeNumber from semantic_type_components stc join semantic_types st on (stc.semanticType = st.expandedForm)) st
WHERE c.terminology = :terminology
AND c.id = cstc.concepts_id
AND cstc.semanticTypes_id = st.id) t1
JOIN 
(select c.id conceptId, st.id styId, st.treeNumber from concepts c, concepts_semantic_type_components cstc, (select stc.id, st.treeNumber from semantic_type_components stc join semantic_types st on (stc.semanticType = st.expandedForm)) st
WHERE c.terminology = :terminology
AND c.id = cstc.concepts_id
AND cstc.semanticTypes_id = st.id) t2
on  (t1.conceptId = t2.conceptId and t1.styId != t2.styId and t1.treeNumber != t2.treeNumber)
WHERE t2.treeNumber like concat(t1.treeNumber, '.%')

  • sfo_lfo: Short form in one concept, long form in another

    Code Block
    --ShortForm/LongForm ofare what object?

  • deleted_cui: CUIs that are going away - will need bequeathal rel

    Code Block
     
unpublishable NCIMTH concepts?

NON-required bins

    rxnorm_merge: RXCUI Merges

    • Same as nci_merge but with RXNORM

      Code Block
      select a.id clusterId, a.id conceptId 
from concepts a, concepts_atoms b, atoms c 
where a.terminology = :terminology   
and a.id = b.concepts_id   and b.atoms_id = c.id    
and c.terminology='RXNORM'  
group by a.id 
having count(distinct c.conceptId)>1
  • cbo_merge: Merged SCUI current version CBO atoms

    • Same as nci_merge but with CBO

      Code Block
      select a.id clusterId, a.id conceptId 
from concepts a, concepts_atoms b, atoms c 
where a.terminology = :terminology   
and a.id = b.concepts_id   and b.atoms_id = c.id    
and c.terminology='CBO'  
group by a.id 
having count(distinct c.conceptId)>1

  • mdr_merge: Merged SDUI current version MDR atoms

    Code Block
    SELECT 
    c.id clusterId, c.id conceptId
FROM
    concepts c,
    concepts_atoms ca,
    atoms a
WHERE
    c.terminology = :terminologyrelated atoms
  -- These live in the DB as "SY" atom relationships with RELA value either equal to "expanded_form_of" or starting with "mth_" and ending with "_form_of"
 
SELECT 
    c1.id conceptId1, c2.id conceptId2
FROM
    concepts c1,
    concepts_atoms ca1,
    atoms a1,
    concepts c2,
    concepts_atoms ca2,
    atoms a2,
    (SELECT 
        ar.from_id, ar.to_id
    FROM
        atom_relationships ar
    WHERE
        publishable = TRUE
            AND relationshipType = 'SY'
            AND c.id(additionalRelationshipType = ca.concepts_id'expanded_form_of'
         AND ca.atoms_id = a.idOR additionalRelationshipType LIKE 'mth_%_form_of')) sfoLfoRels
WHERE
   AND ac1.terminology = 'MDRNCIMTH'
 GROUP BY c.id HAVING COUNT(DISTINCT a.descriptorId) > 1

    pdq_merge: Merged SDUI current version PDQ atoms

    Code Block
    SELECT 
    c.id clusterId, c.id conceptId
FROM AND c2.terminology = 'NCIMTH'
     concepts c,  AND c1.id = ca1.concepts_atomsid
ca,     atoms a WHERE AND    c.terminologyca1.atoms_id = :terminologya1.id
        AND cc2.id = caca2.concepts_id
        AND caca2.atoms_id = aa2.id
        AND aa1.terminologyid = 'PDQ'
GROUP BY c.id
HAVING COUNT(DISTINCT a.descriptorId) > 1
  • sct_sepfnpt: SNOMED concept clusters where the FN and PT terms are separated
    • DUPLICATE of above

  • rxnorm_split: RXCUI splits

     

  • nci_pdq_merge: Concepts containing current version NCI and current version PDQ atoms

    Code Block
    SELECT DISTINCT
    c.id conceptId
FROM
    atoms a,
    atoms a1,
    concepts c,
    concepts c1,
    concepts_atoms ca,
    concepts_atoms ca1,
    terminologies t,
    terminologies t1
WHERE
    c.terminology = :terminology
        AND c1.terminology = :terminology
        AND c.id = ca sfoLfoRels.from_id
        AND a2.id = sfoLfoRels.to_id
        AND c1.id != c2.id

  • deleted_cui: CUIs that are going away - will need bequeathal rel

    Code Block
     -- Skip for now
unpublishable NCIMTH concepts?

NON-required bins

  • rxnorm_merge: RXCUI Merges

    • Same as nci_merge but with RXNORM

      Code Block
      select a.id clusterId, a.id conceptId 
from concepts a, concepts_atoms b, atoms c 
where a.terminology = :terminology   
and a.id = b.concepts_id   and b.atoms_id = c.id    
and c.terminology='RXNORM'  
group by a.id 
having count(distinct c.conceptId)>1

  • cbo_merge: Merged SCUI current version CBO atoms

    • Same as nci_merge but with CBO

      Code Block
      select a.id clusterId, a.id conceptId 
from concepts a, concepts_atoms b, atoms c 
where a.terminology = :terminology   
and a.id = b.concepts_id
    AND ca
    • and b.atoms_id =
    a
    • c.id    
and c.terminology='CBO'
    AND
    • group
    c1.id = ca1.concepts_
    • by a.id 
having count(distinct c.conceptId)>1

  • mdr_merge: Merged SDUI current version MDR atoms

    Code Block
    SELECT 
   AND ca1c.atoms_id =clusterId, a1c.id conceptId
FROM
    concepts c,
AND a.terminology = 'NCI' concepts_atoms ca,
    atoms a
WHERE
   AND ac.terminology = t.:terminology
        AND ac.versionid = tca.versionconcepts_id
        AND tca.currentatoms_id = TRUEa.id
        AND a1a.terminology = 'PDQMDR'
GROUP BY c.id
HAVING     AND a1.terminology = t1.terminologyCOUNT(DISTINCT a.descriptorId) > 1

  • pdq_merge: Merged SDUI current version PDQ atoms

    Code Block
    SELECT 
    c.id clusterId, c.id ANDconceptId
a1.versionFROM
= t1.version   concepts c,
    AND t1.current = TRUEconcepts_atoms ca,
    atoms a
WHERE
   AND c.idterminology = c1.id

    nci_sct_merge: Concepts containing current version NCI and current version SNOMEDCT atoms

    Code Block
    SELECT DISTINCT:terminology
        AND c.id conceptId
FROM
= ca.concepts_id
   atoms a,    AND atoms a1,
    concepts c,
    concepts c1,
    concepts_atoms ca,
    concepts_atoms ca1,ca.atoms_id = a.id
     terminologies t,  AND a.terminology = terminologies'PDQ'
t1
WHERE
   GROUP BY c.terminologyid
= :terminology
   HAVING COUNT(DISTINCT a.descriptorId) > 1
  • sct_sepfnpt: SNOMED concept clusters where the FN and PT terms are separated
    • DUPLICATE of above

  • rxnorm_split: RXCUI splits

     

  • nci_pdq_merge: Concepts containing current version NCI and current version PDQ atoms

    Code Block
    SELECT DISTINCT
    AND c1c.terminologyid =conceptId
:terminologyFROM
    atoms a,
  AND c.id = ca.concepts_idatoms a1,
    concepts c,
  AND ca.atoms_id = a.idconcepts c1,
    concepts_atoms ca,
  AND c1.id = ca1.concepts_idatoms ca1,
       AND ca1.atoms_id = a1.idterminologies t,
    terminologies t1
WHERE
   AND ac.terminology = 'NCI':terminology
        AND ac1.terminology = t.:terminology
        AND ac.versionid = tca.versionconcepts_id
        AND t.currentca.atoms_id = a.id
        AND c1.id = TRUEca1.concepts_id
        AND ca1.atoms_id = a1.id
        AND a.terminology = 'SNOMEDCT_USNCI'
        AND a1a.terminology = t1t.terminology
        AND a1a.version = t1t.version
        AND t1t.current = TRUE
        AND ca1.idterminology = c1.id

    ambig_no_pn: Ambiguous concepts where at least one has no MTH/PN nor NCIMTH/PN (sepstring)

    Code Block
    --Ambiguous concept = concepts that share atoms that have the same case-insensitive name
-- This is used by several of the below queries, so it should be created as a VIEW
 
CREATE VIEW ambig_concepts AS 'PDQ'
        AND a1.terminology = t1.terminology
        AND a1.version = t1.version
        AND t1.current = TRUE
       SELECT DISTINCTAND c.id = c1.id

  • nci_sct_merge: Concepts containing current version NCI and current version SNOMEDCT atoms

    Code Block
    SELECT DISTINCT
    c1c.id conceptId
conceptId1, c2.id conceptId2FROM
    atoms a,
 FROM   atoms a1,
    concepts c1c,
    concepts c1,
    concepts_atoms ca1ca,
        concepts_atoms a1ca1,
    terminologies t,
  concepts c2, terminologies t1
WHERE
     concepts_atoms ca2,c.terminology = :terminology
        atomsAND a2c1.terminology = :terminology
  WHERE      AND   c1c.terminologyid = 'NCIMTH'
 ca.concepts_id
          AND c2ca.terminologyatoms_id = 'NCIMTH'
a.id
           AND c1.id = ca1.concepts_id
 
          AND ca1.atoms_id = a1.id
  
         AND c2a.idterminology = ca2.concepts_id'NCI'
        AND a.terminology = t.terminology
AND ca2.atoms_id = a2.id     AND a.version = t.version
    AND c1.id != c2.id AND t.current = TRUE
        AND a1.lowerNameHashterminology = a2.lowerNameHash
'SNOMEDCT_US'
           AND a1.idterminology != a2.id
   t1.terminology
        AND a1.publishableversion = TRUEt1.version
        AND t1.current = TRUE
        AND a2c.publishableid = TRUEc1.id

  • ambig_no_pn: Ambiguous concepts where at least one has no MTH/PN nor NCIMTH/PN (sepstring)

    Code Block
    --UseAmbiguous Pre-programmedconcept View= ambig_concepts  that SELECTshare atoms that have the same conceptId1, conceptId2
FROM
    ambig_concepts
WHERE
    NOT ambig_concepts.conceptId1 IN (SELECT 
 case-insensitive name
-- This is used by several of the below queries, so it should be created as a VIEW
 
CREATE VIEW ambig_concepts AS
    SELECT DISTINCT
        c1.id conceptId1, cac2.concepts_id conceptId2
    FROM
     FROM   concepts c1,
        concepts_atoms caca1,
        atoms a1,
     atoms a  concepts c2,
     WHERE   concepts_atoms ca2,
        ca.atoms_id =a2
a.id    WHERE
            AND ac1.terminology IN= ('NCIMTH' , 'MTH')
   
            AND ac2.termTypeterminology = 'PN')NCIMTH'
            AND NOT ambig_concepts.conceptId2 IN (SELECTc1.id = ca1.concepts_id
            AND caca1.conceptsatoms_id = a1.id
      FROM      AND c2.id      = ca2.concepts_atomsid
ca,            AND ca2.atoms a_id = a2.id
            AND c1.id != c2.id
WHERE            AND caa1.atoms_idlowerNameHash = a.ida2.lowerNameHash
            AND a1.id != a2.id
      AND a.terminology IN ('NCIMTH' , 'MTH') AND a1.publishable = TRUE
            AND aa2.termTypepublishable = 'PN')
    ambig_no_rel: Ambiguous concepts that lack an approved REL
    TRUE
    Code Block
    --Use Pre-programmed View ambig_concepts
 
SELECT 
    conceptId1, conceptId2
FROM
    ambig_concepts
WHERE
    NOT (conceptId1 , conceptId2) ambig_concepts.conceptId1 IN (SELECT 
            cr.from_id, cr.toca.concepts_id
        FROM
            concept_Relationships crconcepts_atoms ca,
            atoms a
        WHERE
            crca.publishableatoms_id = a.id
   TRUE             AND cra.workflowStatusterminology inIN ('READY_FOR_PUBLICATIONNCIMTH' , 'PUBLISHED'))
    pn_pn_ambig: Identical (same SUI) PN's in multiple concepts

    multiple_pn: Concepts with multiple MTH/PN atoms

    Code Block
    SELECT DISTINCT
MTH')
   c.id conceptId FROM     concepts c,     atomsAND a,.termType = 'PN')
  concepts_atoms ca WHERE    AND c.terminology = :terminologyNOT ambig_concepts.conceptId2 IN (SELECT 
         AND c.id = ca.concepts_id
        FROM
 AND ca.atoms_id = a.id        concepts_atoms ANDca,
a.terminology in ('MTH', 'NCIMTH')         ANDatoms a.termType = 'PN'
GROUP BY c.id
HAVING COUNT(c.id) > 1
    pn_no_ambig: Concept has MTH/PN atom but no ambiguous string
    ambig_pn: MTH/PN atom is ambiguous but has no matching ambiguous string

    pn_orphan: MTH/PNs on their own

    Code Block
    --Concepts whose only publishable atoms are MTH/PN or NCIMTH/PN
        WHERE
            ca.atoms_id = a.id
                AND a.terminology IN ('NCIMTH' , 'MTH')
                AND a.termType = 'PN')

  • ambig_no_rel: Ambiguous concepts that lack an approved REL

    Code Block
    --Use Pre-programmed View ambig_concepts

SELECT 
    conceptId1, conceptId2
FROM
    ambig_concepts
WHERE
    NOT (conceptId1 , conceptId2) IN (SELECT 
            cr.from_id, cr.to_id
        FROM
            concept_Relationships cr
        WHERE
            cr.publishable = TRUE
            AND cr.workflowStatus in ('READY_FOR_PUBLICATION','PUBLISHED'))

  • pn_pn_ambig: Identical (same SUI) PN's in multiple concepts

    Code Block
    SELECT 
    c1.id conceptId1, c2.id conceptId2
FROM
    concepts c1,
    concepts c2,
    concepts_atoms ca1,
    concepts_atoms ca2,
    (SELECT 
        a1.id atomId1, a2.id atomId2
    FROM
        atoms a1, atoms a2
    WHERE
        a1.termType = 'PN'
            AND a2.termType = 'PN'
            AND a1.stringClassId = a2.stringClassId
            AND a1.id != a2.id) identicalPNAtoms
WHERE
    c1.terminology = :terminology
        AND c2.terminology = :terminology
        AND c1.id = ca1.concepts_id
        AND c2.id = ca2.concepts_id
        AND ca1.atoms_id = identicalPNAtoms.atomId1
        AND ca2.atoms_id = identicalPNAtoms.atomId2
        AND c1.id != c2.id

  • multiple_pn: Concepts with multiple MTH/PN atoms

    Code Block
    SELECT DISTINCT
    c.id conceptId
FROM
    concepts c,
    atoms a,
    concepts_atoms ca
WHERE
    c.terminology = :terminology
        AND c.id = ca.concepts_id
        AND ca.atoms_id = a.id
        AND a.terminology in ('MTH', 'NCIMTH')
        AND a.termType = 'PN'
GROUP BY c.id
HAVING COUNT(c.id) > 1

  • pn_no_ambig: Concept has MTH/PN atom but no ambiguous string

  • ambig_pn: MTH/PN atom is ambiguous but has no matching ambiguous string

  • pn_orphan: MTH/PNs on their own

    Code Block
    --Concepts whose only publishable atoms are MTH/PN or NCIMTH/PN
 
SELECT 
    c.id conceptId
FROM
    concepts c,
    concepts_atoms ca
WHERE
    c.terminology = :terminology
        AND c.id = ca.concepts_id
        AND ca.concepts_id IN (SELECT 
            ca.concepts_id
        FROM
            concepts_atoms ca,
            atoms a
        WHERE
            ca.atoms_id = a.id
                AND a.terminology IN ('MTH' , 'NCIMTH')
                AND a.termType = 'PN'
                AND a.publishable = true)
GROUP BY ca.concepts_id
HAVING COUNT(DISTINCT ca.atoms_id) = 1

  • nosty: No STY

    Code Block
    SELECT DISTINCT
    c.id conceptId
FROM
    concepts c
        LEFT JOIN
    concepts_semantic_type_components cs ON (c.id = cs.concepts_id)
WHERE
	c.terminology= :terminology
    AND cs.semanticTypes_id IS NULL

  • missing_sty: Reviewed concepts without releasable Semantic Types

    Code Block
    SELECT DISTINCT
    c.id conceptId
FROM
    concepts c
WHERE
    c.terminology = :terminology
        AND c.publishable = TRUE
        AND c.workflowStatus IN ('READY_FOR_PUBLICATION' , 'PUBLISHED')
        AND NOT c.id IN (SELECT 
            c.id
        FROM
            concepts c,
            concepts_semantic_type_components cs,
            semantic_type_components s
        WHERE
            c.terminology = :terminology
                AND c.id = cs.concepts_id
                AND cs.semanticTypes_id = s.id
                AND s.publishable = TRUE)

  • cbo_chem: Current version CBO concepts with CHEM STYs

    Code Block
    -- to find chemical semantic types
(select distinct semanticTypeCategoryMap sty
from projects a, ProjectJpa_semanticTypeCategoryMap b
where a.id = b.ProjectJpa_id
  and semanticTypeCategoryMap_KEY = 'chemical'
  and a.terminology = :terminology);
 
-- Rick update 3/2/2017
(select distinct semanticTypeCategoryMap_KEY sty
from projects a, ProjectJpa_semanticTypeCategoryMap b
where a.id = b.ProjectJpa_id
  and semanticTypeCategoryMap = 'chem'
  and a.terminology = :terminology)
 
 



  • go_chem: Current version GO concepts with CHEM STYs

    Code Block
    SELECT DISTINCT
    c.id conceptId
FROM
    concepts c,
    concepts_semantic_type_components cs,
    semantic_type_components s,
    terminologies t
WHERE
    c.terminology = 'GO'
        AND c.terminology = t.terminology
        AND c.version = t.version
        AND t.current = TRUE
        AND c.id = cs.concepts_id
        AND cs.semanticTypes_id = s.id
        AND s.semanticType IN ((SELECT DISTINCT
            semanticTypeCategoryMap_KEY sty
        FROM
            projects a,
            ProjectJpa_semanticTypeCategoryMap b
        WHERE
            a.id = b.ProjectJpa_id
                AND semanticTypeCategoryMap = 'chem'
                AND a.terminology = :terminology))

  • mdr_chem: Current version MDR concepts with CHEM STYs

    • See above

  • true_orphan: Concepts with no releaseable relationships to any other concept

    Code Block
    SELECT 
    c.id conceptId
FROM
    concepts c
WHERE
    c.terminology = :terminology
        AND NOT c.id IN (SELECT 
            c.id
        FROM
            concepts c,
            concept_relationships cr
        WHERE
            cr.from_id = c.id
                AND cr.publishable = TRUE)

  • deleted_cui_split: Complex split/merge case CUIs that are going away - will need bequeathal

  • split_demotions: Demotions overlapping with a bad merge that was split
  • nci_pt_mrg: Merged current version NCI/PT atoms
  • mxsuppr: Concepts with identical LUI atoms that have mixed suppressibility

...