Versions Compared

Version Old Version 1 New Version Current
Changes made by

bcarlsen (Unlicensed)

bcarlsen (Unlicensed)

Saved on

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

...

Describes how to set up and deploy the application to the IHSTDO WCI term-server.com server.

Diagram

Image Added

Details

...

  1. Log into the server 162server 10.243230.215174.7616
    1. You'll need the ihtsdo user password (uat3#Map for UAT, prod3#Map for prod)to be able to sudo su - to root and ec2-tomcat.

  2. Install and configure software as root (nginx, tomcat7 , libapache2_mod_jk, (and webapps), mysql-server-5.5 (and client), Java/jdk (including devel), maven, git, svn, etc)
    You'll need the root user password.

    1. Edit the /etc/init.d script to comment out the top part that forces running as root

    2. Edit the /etc/init.d script to use this so the ec2-user can write the pid file:

       
      Code Block
      CATALINA_PID="/var/lib/tomcat7/$NAME.pid"

# also change all instances of /var/run to /var/lib/tomcat7

    3. Set permissions of the tomcat installation to the "ihtsdoec2-tomcat"user:

       

       
      chown -R ihtsdo /var/log/tomcat7
      chgrp -R ihtsdo /var/log/tomcat7
      chown -R ihtsdo /var/cache/tomcat7
      chgrp -R ihtsdo /var/cache/tomcat7
      chown -R ihtsdo  /var/lib/tomcat7
      chgrp -R ihtsdo  /var/lib/tomcat7
       
      Edit
      Code Block
      # for each linked directory in /usr/share/tomcat7 d
chown -R ec2-tomcat $dir
chgrp -R ec2-user $dir


Edit /etc/init.d/tomcat7 to use the following
      
*
      TOMCAT7
       TOMCAT_USER=
      ihtsdo
      * TOMCAT7_GROUP=ihtsdo
       
      Edit /etc/default/tomcat7
      * TOMCAT7_USER=ihtsdo
      * TOMCAT7_GROUP=ihtsdo
      ** This must be done while tomcat is NOT running.
      See ~root/README.txt
      ec2-tomcat
* TOMCAT_GROUP=ec2-user


Also edit /etc/tomcat7.conf
* CATALINA_PID=/var/lib/tomcat7/tomcat7.pid
* TOMCAT_USER=ec2-tomcat
* TOMCAT_GROUP=ec2-user


Make sure all the "chown" commands in the script change to group to $TOMCAT_USER

       

      NOTE: the tomcat user can start/stop the server with "/usr/sbin/tomcat7 start/stop".
      HOWEVER: if the process was started by root, it will likely have to be killed by root first..  we're working on this. 

       

  3. Tomcat must be configured to pass "run.config.ts" environment variable to the application
    1. One way to do this is to invoke the java command with -Drun.config.ts=~/homeconfig/ihtsdo/data/config.properties
      (e.g. edit this in /etc/init.dusr/share/tomcat7/conf/tomcat7.conf and add to JAVA_OPTS)
    2. Also add -Xmx2500M to JAVA_OPTS in the tomcat startup script.
    3. NOTE: development environments must also be configured as such

  4. Tomcat must be configured to support POST requests of size > 2MB
    1. Edit server.xml and add  maxPostSize="50000000" to each <Connector> element.

  5. Tomcat must be configured to support following of links.  Make sure any Context tags in the configuration contain  allowLinking="true".
    1. Edit context.xml to add this parameter to the Context tag.

  6. Make sure you have MAVEN_OPTS set 
    e.g. -XX:MaxPermSize=512m -Xmx3072m

    To enable tomcat manager, add this to tomcat-users.xml file:

    Code Block
    <role rolename="manager-gui" />
<user name="tomcat" password="********" roles="manager-gui" />
<role rolename="protected" />
  7. Nginx Configuration
    1. Install the SSL keys as root in the appropriate places
    2. Edit /etc/nginx/sites-available/mapping.ihtsdotools.org (see the webapp/src/main/nginx directory for configurations).
    3. Link (if SSH needed)
    4. Add a file "snomed.term-server.com.conf" to /etc/nginx/sites-enabled/mapping.ihtsdotools.org to the file aboveconf.d

    5. Add a Valve to tomcat to handle proxy headers:

      Code Block
              <Valve className="org.apache.catalina.valves.RemoteIpValve"
               remoteIpHeader="x-forwarded-for"
               remoteIpProxiesHeader="x-forwarded-by"
               protocolHeader="x-forwarded-proto"/>
    6. Edit /etc/nginx/nginx.conf and add "client_max_body_size 50M;" to the "http" context.
    7. See the attachment for more details/etc/nginx/nginx.conf contains the master configuration - see for details
    8. For more info click here
       

  8. MySQL Configuration

    1. When configuring the database URL in config.properties, use ?autoconnect=true to avoid connection issues

    2. Also (as root) edit the my.cnf file for your platform and add this to the mysqld section:

      Code Block
      [mysqld]
...
wait_timeout = 230400
interactive_timeout = 230400

    3. It may also be worth considering using the innodb table per file option:

      Code Block
      [mysqld]
...
innodb_file_per_table

      This is useful to better manage the overall size of the database because individual tables can then be optimized rather than having to dump/reload the entire database.

    4. Consider other optimizations to mysql parameters (e.g. http://www.percona.com/blog/2014/01/28/10-mysql-settings-to-tune-after-installation/).

       

  9. As the MySQL root user, create an OTF user with the mysql client:

     
    Code Block
    CREATE USER 'otf'@'localhost' IDENTIFIED BY 'otfpwd';
    
GRANT ALL privileges ON *.* TO 'otf'@'localhost';

  10. As the MySQL root user, create a database in the mysql client:

     
    Code Block
    CREATE
    DATABASE mappingservicedb
      DEFAULT CHARACTER SET utf8
      DEFAULT COLLATE 
     DATABASE tsdb
  DEFAULT CHARACTER SET utf8
  DEFAULT COLLATE utf8_general_ci;
    
CREATE
    DATABASE mappingservicedbtest
      DEFAULT CHARACTER SET utf8
      DEFAULT COLLATE 
     DATABASE tsdbtest
  DEFAULT CHARACTER SET utf8
  DEFAULT COLLATE utf8_general_ci;
    
GRANT ALL ON
    mappingservicedb
     tsdb.* TO 'otf';
    
GRANT ALL ON
    mappingservicedbtest
     tsdbtest.* TO 'otf';

  11. Get code

     

    cd
    mkdir code
    cd code
    git clone

    Create "code", "config", and "data" directories

    1. ~/code - for the code

    2. ~/data - for the data files (only needed on initial load)

    3. ~/config - for the configuration files and configurable admin scripts:

      Code Block
      cd
mkdir code
mkdir data
mkdir config

  12. Get code:

    Code Block
    cd ~/code
git clone https://github.com/WestCoastInformatics/
    OTF
    SNOMED-
    Mapping
    Terminology-
    Service
    Server.git .
     
    # switch to dev branch - for UAT
    git checkout develop
     
    # switch to dev branch - for PROD
    git checkout master
     
    mvn -DskipTests=false clean install

    Get data.

     

    cd
    mkdir data
    git clone ssh://csfe.aceworkspace.net:29418/mapping data
    mkdir data/doc
    mkdir data/doc/archive
    chmod -R ga+rwx data/doc

  13. Prepare the environment (set your shell up with these ENV variables):

    Code Block
    JAVA_HOME=/usr/lib/jvm/jre-1.7.0-openjdk.x86_64
MAVEN_OPTS="-XX:MaxPermSize=512m -Xmx3000M"



  14. Build all of the code, making sure to choose the correct "config.artifactId" for your environment.  Shown is "prod" configuration:

    Code Block
    cd ~/code
mvn -Dconfig.artifactId=term-server-config-prod clean install

  15. Prepare the data:

    Code Block
    cd ~/data
unzip ~/code/config/term-server-config.*.zip

  16. Prepare the tomcat indexes location:

     
    Code Block
    # To clean up the indexes do this (as root)
    
/bin/rm -rf /var/lib/tomcat7/indexes
    
mkdir /var/lib/tomcat7/indexes
    mkdir /var/lib/tomcat7/indexes/lucene
    mkdir /var/lib/tomcat7/indexes/lucene/indexes
    chmod -R 
    
chmod -R ga+rwx /var/lib/tomcat7/indexes
    
chown -R
    ihtsdo /var/lib/tomcat7/indexeschgrp -R ihtsdo 
     ec2-tomcat:ec2-user /var/lib/tomcat7/indexes
     
    # Ensure the 
    
 
# Ensure the config.properties file used to run the application has this setting
    
hibernate.search.default.indexBase=/var/lib/tomcat7/indexes
    /lucene
    /
    indexes
     
  17. Create and Load the data
     
    cd ~/code/admin/loader/scripts
    load_prod.csh >&! load_prod.log
     
    Note also that TreePositions will be automatically calculated when loading a terminology.  To recompute SNOMED tree positions without reloading terminology, use:
     
    cd ~/code/admin/loader
    mvn -Drun.config=prod -P SNOMEDCT-treepos
  18. Stop tomcat, clear cache,and remove the currently deployed .war files (and associated folders).
     
    # Stop the tomcat7 server
    service tomcat7 stop
    /bin/rm -rf /var/lib/tomcat7/work/Catalina/localhost/mapping-rest
    /bin/rm -rf /var/lib/tomcat7/webapps/mapping-rest
    /bin/rm -rf /var/lib/tomcat7/webapps/ROOT
    /bin/rm -rf /var/lib/tomcat7/webapps/mapping-rest.war
    /bin/rm -rf /var/lib/tomcat7/webapps/ROOT.war
  19. Update the database (this needs to go into the reload page):
     
    mvn clean install -Drun.config=/home/ihtsdo/config/config.properties -Dhibernate.hbm2ddl.auto=update
  20. Update file modification date to prevent caching issues.
     
    cd ~/code
    set date = `date +%Y%m%d%H%M`
    touch -t $date `find . -name "*"`
  21. Build Rest and deploy war
     
    cd ~/code/rest
    mvn -Drun.config=$OTF_MAPPING_CONFIG clean install
    /bin/cp -f target/mapping-rest*war /var/lib/tomcat7/webapps/mapping-rest.war
  22. Build Webapp war
     
    cd ~/code/webapp
    mvn -Drun.config=$OTF_MAPPING_CONFIG clean install
    /bin/cp -f target/mapping-webapp*war 
    Unpack the config artifact corresponding to your environment The example shown below is for "prod".
     Code Block
    cd ~/config
unzip ~/code/config/prod/target/term-server-config-prod.*.zip
  23. Edit the configuration file and scripts.
    1. Edit config.properties and replace the "EDIT_THIS" entries with real values (mostly passwords, etc).
    2. Edit the mail notification settings to list email addresses of those admins or maintenance staff that should be notified of errors and/or automation outcomes.
    3. Edit the scripts in bin/ as needed for this environment (default setup requires no changes).|
       
  24. Edit the load script for your environment. Run it to load the data:
     Code Block
    cd ~/config/bin
chmod 775 *sh
./load.csh >&! load.log
  25. Deploy war:
     Code Block
    /bin/cp -f ~/code/rest/target/term-server-rest*war /var/lib/tomcat7/webapps/
    ROOT
    term-server-rest.war
  26. Start service and link "doc" directory.
     
    :
     Code Block
    service tomcat7 start
     
    # wait for deploy to create the ROOT directory
    sleep 5
    # link doc directory
    cd /var/lib/tomcat7/webapps/ROOT
    ln -s /home/ihtsdo/data/doc
    NOW, if deploying to PROD, create a github pull request.
  27. Wire tomcat to redirect to the home URL.  In the webapp root find the ROOT application and replace the index.html page. e.g.
    NOTE:  DO NOT do this step anymore as the application is now deployed as the root war
     
    <?xml version="1.0" encoding="ISO-8859-1"?>
    <!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Strict//EN"
       "http://www.w3.org/TR/xhtml1/DTD/xhtml1-strict.dtd">
    <html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en" lang="en">
    <head>
        <meta http-equiv="refresh" content="0; url=/mapping-webapp/app/index.html#/" />
        <title>IHTSDO Mapping Tool - redirect</title>
    </head>
    <Body>
    </body>
    </html>
 
Complete Data Load
 
Getting Backups from PROD to UAT
A daily backup tgz file exists on the prod server at
  • /var/backup/data.tgz
Alternatively, you could log into the uat and use this command
 
sudo aws s3 ls mapping.backup.ihtsdo
 
 
This will produce a list such as the following (just run):
2014-09-28 01:02:47 343656836 dbbackup_mappingservicedb_201409280100.bak.gz
2014-09-29 01:02:48 343750530 dbbackup_mappingservicedb_201409290100.bak.gz
2014-09-30 01:02:49 343867437 dbbackup_mappingservicedb_201409300100.bak.gz
2014-10-01 01:02:46 343892129 dbbackup_mappingservicedb_201410010100.bak.gz
2014-10-02 01:02:46 349749641 dbbackup_mappingservicedb_201410020100.bak.gz
2014-10-03 01:04:15 349779484 dbbackup_mappingservicedb_201410030100.bak.gz
Choose the backup you want to copy (it keeps the most recent 6 days) and do the following (this example taking the most recent backup):
 
sudo aws s3 cp s3://mapping.backup.ihtsdo/dbbackup_mappingservicedb_201410030100.bak.gz ~/
 
 
This  will copy to ~/ (or wherever you want by changing that location at the end of the command).  Then gunzip the file.
  1. Run the integration tests (only do this if you know what you're doing):
     Code Block
    cd
cd code/integration-test
mvn -DskipTests=false -Drun.config.ts.test=~/config/config-test.properties test
  • n/a