Loading a Database with Full 201610 Data

Overview

Instructions on loading a full database.

Details

1. Create a separate working dir, e.g. c:/ncifull
   1. Create config/config.properties file for this work -e.g. config.properties
      1. set all directories to c:/ncifull instead of c:/umlsserver
   2. Set up a second "server" in Eclipse to run with this config file.
   3. Set up secondary run configurations that use this config.properties file
2. Truncate your db (or prepare a fresh db)
   1. Make sure you are using collation utf8_bin
   2. ALTER DATABASE ncifulldb DEFAULT CHARACTER SET utf8 COLLATE utf_bin
3. Pull and build the project and make sure your config file is completely up to date
4. Clear your indexes directory of all files
5. Download the data from https://wci1.s3.amazonaws.com/NCI/umls.sql.gz
6. gunzip the file (may require cygwin)
7. Import the file into your database (with MySQL workbench → see the Management/Data Import tool)
8. Reindex your database (run the Reindex profile on the admin/lucene project)
   1. NOTE: make sure your indexed.objects property is blank.

With that, you should be able to build and deploy a server.

NOTE: on certain unix environments MySQL table names are case sensitive and will need to be corrected (ask BAC if needed)

Scale Testing Insertions

Full data for insertions can be downloaded from S3

• Download the "full" SRC data - https://wci1.s3.amazonaws.com/NCI/NCI-srcDataDir-full.zip
• Unzip this into c:/umlsserver/data
• Edit config-full.properties so that source.data.dir points to c:/umlsserver/data
• Run a server, and perform NCI, SNOMEDCT_US, or MTH insertion.

Scale Testing Release

• Download the "mr" package data - https://wci1.s3.amazonaws.com/NCI/NCI-srcDataDir-full.zip
• Unzip this into c:/umlsserver/data
• Edit config-full.properties so that source.data.dir points to c:/umlsserver/data
• Run a server, and perform NCI, SNOMEDCT_US, or MTH insertion.

References/Links

• n/a