SPRINT 2 - Finish 5/15/2015

Overview

Sprint to continuing to develop core features of the application

Fleshing out APIs

***Flesh out "examples" project to 10 examples
- Address "toString" methods of the main data types.
***Change "TerminologyVersion" => "Version" (terminologyVersion => version, "terminology version" => version)
- *java, *html, *js, *properties
- This requires all features checked in and no new dev underway. (shoot for Friday 5/29)
Handle blank query scenario in "findConceptsForQuery", etc..http://stackoverflow.com/questions/10151992/rest-how-to-pass-empty-path-parameter
DONE: Double-check that all rest services always close the service (e.g. in "finally")
PG (in progress): Mobile friendly site

DONE: Lucene enhancements
- DONE: Autocomplete/spelling correction
Content service rest
- Implement searching for tree positions by ancestor path.
- DONE: Implement getConcept, getDescriptor, getCode, getLexicalClass, and getStringClass
- DONE: Implement findConceptsForQuery (ignore branching for now) and related methods.
- DONE: Implement rest client as well
- DONE: Implement methods for accessing subsets as well
  - GraphReoslutionHandler:
    - public resolve(Subset)
    - public resolve(SubsetMember)
  - Call the resolver on the ContentServiceRest services for subset/subsetmember (like it is used in getConcept/findConcepts...)
  - DefaultGraphResolutionHandler/UmlsGraphResolutionHandler
    - for subsets:
      - resolve name
      - "skip" subset members
      - resolve attributes
    - for subset members
      - resolve member.getName()
      - resolve attributes
- DONE: Begin developing integration tests to validate the data (could start with content service tests)
- DONE: later : get relationships for a concept - e.g. CUI relationships
  - ContentServiceRest getDeepRelationshipsForConcept(String terminologyId/terminolgy/Version, PfsParameter pfs, auth token)
- DONE: autocompletion methods (BAC)
- treeposition methods
  - TreeList findTreesForConcept/Code/Descriptor(String terminologyId/terminology/version, boolean includeChildren, PfsParameter pfs)
  - findConcept/Code/DescriptorTreesForQuery(String terminology/version, String query, boolean startAtRoot, PfsParameter pfs)
  - getRootConcepts/Codes/Descriptions
- DONE: Subset services
  - getSubset - by id or terminologyId - NO REST CALL FOR THIS
  - getAtomSubsets - SubsetList
  - getConceptSubsets
  - getSubsetMembersForAtom - takes atom terminology id
  - getSubsetMembersForConcept - takes a concept terminology id
  - findAtomSubsetMembers
  - findConceptSubsetMembers
REST service for a general concept query (e.g. lucene query, HQL query, or SQL query)
- Must start with "SELECT " and not contain ";"
  - provide appropriate error message via exception if not
- Execute the query but check the object type of the first result
  - if doesn't match expected type, fail with an exception.
- Implement for Concept, Code, Descriptor
- Verify hqlQuery must start with
  - SELECT a FROM Concept/Code/DescriptorJpa a (depending on type)
- ContentServiceRest/Impl...
- ContentService/Jpa
  - Find the type of query - apply the PFS appropriately.
  - use the "set timeout" feature of JPA queries.
  - use "createQuery" for HQL
  - use "createNativeQuery" for NATIVE
- Use timeout =
```
query.setHint("javax.persistence.query.timeout", timeout);
```
Domain model
- DONE: Add property chains (to metadata)
- DONE: LexicalClass - set normalized string (needs handler)
- DONE: TreePosition (and tree position computer) - childCt
Semantic search (via search criteria)
- IN PROGRESS: Add criteria for the results being a descendant of X
DONE: UserPreferences - tty list
Websocket (for a WebsocketWorkflowListener)
"smart" RRF loader should support a config file to indicate what level at which definitions should be attached and should handle RXNORM and NCIt concepts.
Updated UI for UMLS
- DONE: Base view around a Concept Report
- DONE: page relationships (and possibly attributes)
- compute/page "contexts" - need to actually compute paths to the root
- DONE: +/- icons for expanding elements (e.g. atoms) that have more info
- UI for "semantic search" and for "general query" search
- terminology lock/unlock - to turn it into a single-terminology browser.
"Marked parent refset" - or in this case for subsets - identify the concept ancestors (e.g. of concept subsets) so they can be labelled in a hierarchical display
- decorate tree position operations when handling browser stuff.
DONE: Startup routines should verify the presence of "DEFAULT" handlers when they are needed to exist
DONE: ContentServiceJpa/Rest
- graph resolver doesn't resolve relationships
- rest call for getting the relationships of a concept/descriptor/etc
Callback services (e.g. for subset members, relationships, or tree positions should always include a "query" and those corresponding objeects should be indexed with their xmlTransient-related fields).
Optimize the handling of "umls metadta handler" with respect to caching all the data. Maybe cache it all immediately upon loading?
- or in a thread
- either that or track which values are actually used at load time. - though caching doesn't require QA.

Fleshing out the API

Switch to a default build of the SAMPLE_2014AB and redo metadata and content tests against that data set. Then we may be able to have more complete hierarchies and some code data to work with.

Testing

DONE: Model objects (content)
DONE: List objects
DONE: Helpers
Metadata service integration test
- add tests for new getTerminology method
Content service integration test
Test conditional envers auditing: http://stackoverflow.com/questions/14250612/conditional-envers-auditing

User Interface Dev

Browser with hierarchy
Basic concept editing (editable report)
Subset management (e.g. “refset” tool)

Admin Tools

RRF Loader
- Finish metadata loading
- Do content loading
RRF Loader - single source

Future Stuff

escg (expression grammar - research)
Use Lucene SynonymFilter with synonym table
Component-Component relationships (between any two components).
Value set definitions (and marking of subset.isValueSet()) and linking to definition? via attribute?
Owl loader, Owl export of DL terminologies (e.g. RF2-loaded SNOMED)
- http://owlapi.sourceforge.net/owled2011_tutorial.pdf
Rdf export (flat)
Classifier (owl interface)
Expression language (based on SNOMED expression constraint grammar)
Sub-branching
- branchResolutionHandler - figures out how to copy and mark branched objects and update indexes - for different branching strategies.
Handle mappings - may be not worth it
Implement an RF2 loader (use DL features)
Implement a ClaML loader
Support semantic network (e.g. sty rels, etc). - probably want to wait for a real ontology - and maybe even load it as such.