...
- DONE (BC): Fix long error header - attach icon not showing, where it displays error is not in the error area.
- DONE PG: Need to index atoms as lowercase string, punctuation replaced by whitespace, no trailing, duplicate, or leading whitespace.
Create a new tokenizer factory for this and define in @AnalyzerDefs in AbstractAtomClass- "Cholsterol.in HDL/Cholesterol.in LDL" => " cholesterol in hdl cholesterol in ldl"
- "ABC def----ghi" => "abc def ghi"
"ABC ////df3948123,.3482932–..xx abc" => "abc df3948123 3482932 xx abc"
Add an AtomJpa method called getNameNorm() that isn't analyzed but returns a lowercased string split on punctuation
- Put algorigthm into ConfigUtility - getIndexerNormalizedString
- String.join(" ","hiv antibody".toLowerCase().split("[ \t-({[)}]_!@#%&*\\:;\"',.?/~+=|<>$`^]")).replaceAll(" +"," ");
- Default search handler can include as a boosted "or" clause in the query, e.g. by 10
- if literalField.endsWith("Sort"), create normField = literalField.replace("Sort","Norm"),
- add an OR clause with this field and the normalized value.
- "HIV Antibody" =>
- (HIV Antibody OR nameNorm:\"hiv antibody\"^10.0 OR nameSort:\"HIV Antibody\"^20.0) AND ...
- "1-2,Anydroxy" =>
- (1-2,Anhydroxy OR nameNorm:\"1 2 anhydroxy\"^10.0 OR nameSort:\"1-2,Anhydroxy\"^20.0) AND ...
- DONE PG Clear filters on new searches or navigating (e.g. atom filters stick around)
- Added function resetPaging(), called on load (instantiates filters and paging)
- getComponentFromTree and getComponentFromType now clear paging on successful component retrieval
- DONE PG Relationship sort order - allow sorting on REL, RELA, or TO(terminology)ID
- DONE PG Color code search results based on log base value -> GREEN for high, YELLOW, RED for low. Based on score indexed to a 1 value.
- Normalize scores with a "log base", e.g. take the raw score (may be 12.5, may be 0.83) Choose a log base, e.g. "5", if the score is > 5, it gets a value of 1, otherwise it is the log of the score with the base (e.g. "5").
- when getScoreMap is called -> every score should be in a range from 0 - 1
- DONE PG: Advanced search - like mapping
- Semantic type - [ picklist ]
- semanticTypes.semanticType:"Clinical Drug"
- term type - [ picklist ]
- atoms.termType:...
- match terminology - [ picklist ] (only for "metathesaurus")
- atoms.terminology:..
- language - [ picklist ]
- atoms.language:...
- Semantic type - [ picklist ]
- DONE PG: support multiple sort fields
- keep pfsParameter sortField
- pfsParameter.get/setSortFields(List<String> fields)
- sortFields overrides sortField
- Handle in IndexUtility where new Sort(...) is. Make sure to handle date & id fields as LONG and other fields as STRING.
- DONE PG Relationship Filter needs to search more efficiently
- DONE e.g. only single word matching currently performed ("600" does not return "600mg" terms)
- Fixed by adding wildcard searches directly from webapp
- DONE Misc. improvements:
- Return to page 1 on change of filter or sort field/sort order
- Add tooltip to sort order icon
- DONE Relationships should be loaded with blank value instead of 0 (RRF loader, line 2151)
- DONE Index toFromName on ConceptRelationshipJpa, CodeRelationshipJpa, DescriptorRelationshipJpa
- QUESTION: Relationship searching is set up to handle inverse cases, and forces a fielded search in DefaultSearchHandler. Is this really what we want from the webapp?
- DONE e.g. only single word matching currently performed ("600" does not return "600mg" terms)
- DONE PG Implement atom class search handler
- DONE Should search name and id fields
- Should only search DONE Search id fields if query contains no white space (i.e. could possibly be an id)Add alphanumeric checks as well (must contain 1 number)single term contains at least one number
- DONE Should throw exception if invoked on a non-AbstractAtomClass
- Current DONE Find reasonable weights:
- Terminology Id -> 4.0 NOTE This is across all fields (boosted expression)
- Includes atom.conceptId, atom.descriptorId, atom.codeId
- Name (Sort) -> 45.0
- Name (Norm) -> 25.0
- Name (by split term) -> 1.0
- Terminology Id -> 4.0 NOTE This is across all fields (boosted expression)
- DONE Split term needs to respect quoted terms, e.g. "a b", "a b" c "d e f"
- DONE PG: Scale down DefaultSearchHandler
- Get rid of acronym expansion
- Get rid of spelling checks
- Get rid of normalized field checking
- Goal: text in, results out, basic filtering/searching features
- Fielded search
- If no results literal search
- If no results wildcard-appended search
- PG: redo header/footer - controllers. (like refset tool)
- PG: Implement Diagramming
- model transformation
- Show ONLY for "description logic terminology"
- Enhancements to RRF preferred name computer
- support multiple UMLS's - e.g. have a high-level terminology/version
- reuse the same default precedence list.
- IN PROGRESS PG: Use directives for reports
- Move all http calls to services.
- DONE PG: Paging controls are no longer showing up for subcategories (e.g. atoms, relationships)
- IN PROGRESS Use route params to dynamically load a "report" (and be able to bookmark URLs)
- Implement routing for terminologyId/terminology/version/type so we can preserve URLs. can even include ?query=... for the query
- Also clean up the way routes work first so we have
- instead of
- This is likely related to the starting URL redirected to https://umls.terminology.tools/#/
- See http://stackoverflow.com/questions/14319967/angularjs-routing-without-the-hash
- e.g. https://umls.terminology.tools/content/CUI/SNOMEDCT/2015_01_31/12738006
- Implement routing for terminologyId/terminology/version/type so we can preserve URLs. can even include ?query=... for the query
- Support opening a concept in a new window (e.g. there's a pointing off arrow icon that opens a new window with a routing URL that shows exactly that concept - then drag/drop between windows can be editing mechanism).
- User Preferences stuff
- Bring over model from Refset tool.
- Add "last query" (e.g. "brain" and whether it's "list" or "tree" mode).
- Add "last report" (e.g. type/ui/terminology/version)
- Remove "void addXXX" and "void removeXXX" methods from model objects. Use getObjects().add/remove(...) only
- e.g. AtomClass doesn't need addAtom or removeAtom
- Rewire any uses of them
- "addXXX" -> "getXXX().add"
- "removeXXX" -> "getXXX().remove"
- same if there is "clear"
- DSS: Model
Mapping, MapRecord, MapEntry (each with attributes) -similar to Subset, SubsetMember except with one more layer.
- MapSet extends ComponentHasAttributes
- name (String)
- type (String)
- fromTerminology/Version: String
- toTerminology/Version: String
- to/fromComplexity (String)
- to/fromExhaustive (String)
- get/setMappings -> use mappedFrom in @OneToMany
- Mapping extends ComponentHasAttributes
- fromTerminologyId, fromIdType
- toTerminologyId, toIdType
- mapSet
- relationshipType, additionalRelationshipType
- subsetId => group
- rank
- rule
- restriction -> rename to 'advice'
- In Jpa class
- Add @XmlTransient methods with @Field
- getFromTermionlogy(), getFromVersion() - call getMapSet().getXXX
- getToTermionlogy(), getToVersion() - call getMapSet().getXXX
- Remove MapObject
- MappingList/Jpa
- ContentService - Jpa
- add/remove/update MapSet/Mapping
- getMapping(Long id)
- getMapping(String terminologyid, terminology, version, branch)
- findMappingsForMapSet(Long mapSetId, query, pfs)
- findMappingsForConcept(Long conceptId, query, pfs)
- call getConcept(conceptId).
- then perform a "from" search based on the terminologyId, terminology, version, and idType (e.g. CONCEPT)
- findMappingsForCode(Long codetId, query, pfs)
- findMappingsForDescriptor(Long descriptorId, query, pfs)
- REST layer - ContentServiceRest (Impl,Client)
- getMapsets
- /mapset/all/{terminology}/{version}
- getMapset
- /mapset/{terminologyId}/{terminology}/{version}
- findMappingsForMapSet(Long mapSetId, query, pfs)
- /mapset/{terminologyId}/{terminology}/{version}/mappings
- findMappingsForXXX(Long xxxId, query, pfs)
- /cui/{terminology}/{version}/{terminologyId}/mappings
- /code/...
- /dui/...
- getMapsets
- Integration tests
- jpa
- REST
- basic test of client layer - e.g. make sure the calls to the server actually work and return something
- Unit Tests
- MapSetJpa, MappingJpa, MappingListJpa
- Loaders
- Then implement this for RRF loader
- MRSAT
Handle in loadMrsat
- Have a class-level Map<String, MapSet> mapSetMap variable
- When encountering a "mapping" attribute, create an entry in this map if it doesn't exist
- use the CUI as the key
- Extrac the mapset from the map, then set the corresponding field
- At the end of loadMrsat, for each mapSet in the variable, call service.addMapSet(...)
MTH_MAPFROMEXHAUSTIVE => mapSet.fromExhaustive
MTH_MAPTOEXHAUSTIVE => mapSet.toExhaustive
MTH_MAPSETCOMPLEXITY => mapSet.complexity
- MTH_MAPSETFROMCOMPLEXITY => mapSet.fromComplexity
- MTH_MAPSETTOCOMPLEXITY => mapSet.toComplexity
MAPSETSID = mapSet.terminologyId
- MAPSETNAME = mapSet.name
MAPSETVERSION = mapSet.mapVersion
- MAPSETFROMVERSION = mapSet.mapVersion
- MAPSETTOVERSION = mapSet.mapVersion
- Add a loadMrmap method
- use mapSetMap when processing mapping entries.
- skip entries where the FROMTYPE or TOTYPE fields cannot be resolved to an IdType enum (IdType.valueOf(...) fails)
- Look up the MapSetCui in mapSetMap, use this for mapping.setMapSet
- MAPSETCUI - used to look up in mapSetMap
- MAPSETSAB - mapSet.terminology
- MAPSUBSETID = mapping.group
- MAPRANK = mapping.rank
- MAPID, MAPSID = mapping attribute, name="MAPID" value = "value of the MAPID field"
- FROMID, FROMSID = mapping attribute
- FROMEXPR = mapping.fromTerminologyId
- FROMTYPE = mapping.fromIdType
- FROMRULE, FROMRES = mapping attribute
- REL,RELA = mapping.relationshipType, additionalRelationshipType
- TOID,TOSID = mapping attribute
- TOEXPR = mapping.toTerminologyId
- TOTYPE = mapping.toIdType
- TORULE,TORES = mapping attribute
- MAPRULE = mapping.rule
- MAPRES = mapping.advice
- MAPTYPE = mapping.type
- MAPATN,MAPATV = mapping attribute
- CVF - n/a
- MRSAT
- Then implement this for RF2 loader (snapshot)
- Then implement this for RF2 loader (delta)
- Then implement this for RRF loader
- Show mappings in UI.
- Show atom-level subset information (e.g. language refset entires in SNOMED. may require a change to the graph resolver).
...