Developer documentation

Contributions and branches

All pull requests should go to the develop branch or a feature branch. If the feature or bugfix will take more than a single PR to complete, it should go to a feature branch in the workspace repo. Contact a maintainer to create a feature branch if this is the case.

Branches:

  • feature branches - work in progress goes here, not stable, tests may not pass.

  • develop - All tests pass. Feature branches are merged here when features are ready for release. Ready for integration testing.

  • master - All tests pass, code is production ready.

develop deploys to ci.kbase.us. Generally speaking, most development would occur on develop, but because most of ci would break if the workspace breaks, develop must be kept stable.

Recompiling the generated code

To compile the Workspace generated code from workspace.spec, run ./gradlew sdkCompile.

Note that:

  • The kb-sdk executable must be in the system path.

  • The WorkspaceServer class compiled in constructor has been commented out in order to use a custom constructor. In order for the compile to succeed, the custom constructor must be commented out and the compiled in constructor uncommented. When the compile is complete the constructors must be switched back.

Release checklist

  • Update the version in docsource/conf.py

    • Use a semantic version in the format X.Y.Z.

  • Update the version in the us.kbase.workspace.version.WorkspaceVersion class.

    • Use a semantic version in the format X.Y.Z-devA, where X.Y.Z is the semantic version and A is the incremental dev version.

  • Update release notes

  • Update documentation if necessary.

  • Ensure tests cover changes. Add new tests if necessary.

  • Run tests against supported versions of MongoDB and dependencies.

  • Merge the feature branch or fork branch to develop.

  • Tag the release in git with the new dev version.

  • When satisfied with CI testing (work with devops here):

    • Change the version to X.Y.Z.

    • Merge develop to master.

    • Tag master with X.Y.Z in git.

Deploying the Workspace Service locally

Todo

This section needs an entire rewrite from scratch with Tomcat as the application server. Currently, the easiest way to run the service locally is via docker compose up -d --build which will start a KBase auth server in testmode and the workspace. If deploying outside a docker container is required, the best option for now is to inspect the Dockerfile and attempt to follow the steps there.

Also, the developer and administrator server startup documentation should be unified.

These instructions are known to work on Ubuntu 16.04 LTS.

  1. Install the dependencies Java8, Apache Ant, pymongo v2.8, GlassFish v3.1.2.2 , mongodb >=v2.6.*, kb-sdk and the KBase Jars directory.

$ sudo pip install pymongo==2.8

The unix shell script (ogs-3.1.2.2-unix.sh) install of GlassFish is simple since the application configuration is also handled during install. Follow the wizard instructions to complete the GlassFish installation. Leave all config values to default values.

The KBase Jars directory must be placed in a directory that is adjacent to the workspace directory.

The rest of this playbook assumes that you have the Glassfish, Mongo and kb-sdk binaries in your system environment path variable.

$ cd [PATH_TO_YOUR_WORKSPACE_DIR]
$ cd ..
$ git clone https://github.com/kbase/jars.git

  1. Start mongodb. Open a new terminal and create a directory for your mongo data (if one does not already exist), then start mongodb.

$ mkdir ~/mongodata
$ mongod --dbpath ~/mongodata

  1. Set up the workspace for deployment in another terminal.

$ cd [PATH_TO_YOUR_WORKSPACE_DIR]

Note

If you are using Oracle Java 8, the javadoc command may throw errors and warnings. Add the following linter argument line to the javadoc command in build.xml to suppress these warnings and errors.

<javadoc access="protected" author="false" classpathref="compile.classpath"
  destdir="${doc}" nodeprecated="false" nodeprecatedlist="false"
  noindex="false" nonavbar="false" notree="false"
  source="1.7" splitindex="true" use="true" version="true">
  <arg line="-Xdoclint:none"/>   <!-- ADD THIS LINE -->
  <link href="http://download.oracle.com/javase/8/docs/api/"/>
  ....
</target>

Then run make.

$ make

Set up a fake kbase directory with a softlink to glassfish within it.

$ cd ../
$ mkdir fakekb
$ cd fakekb
$ ln -s ~/glassfish3
$ gedit glassfish3/glassfish/config/osgi.properties

Add this fix at the end of the osgi.properties file -

# fix for java 8
jre-1.8=${jre-1.7}

Make sure to get latest version of dev-candidate branch from git.

$ cd ../workspace_deluxe
$ git checkout dev-candidate
$ git pull

Configure the service for deployment. The instructions here assume the deployment is tied to the CI environment.

$ cp deploy.cfg.example deploy.cfg
$ gedit deploy.cfg

Make the following changes -

auth-service-url = https://ci.kbase.us/services/auth/api/legacy/KBase/Sessions/Login
auth2-service-url = https://ci.kbase.us/services/auth/
ws-admin = [YOUR_NAME]
# Note: ignore-handle-service does not exist and needs to be added
ignore-handle-service = true

  1. Initialize and start the workspace service. This deployment uses gridFS rather than shock as a file backend and does not support handles to shock nodes in objects, and any attempt to save an object with handles will fail.

$ export KB_DEPLOYMENT_CONFIG=[ABSOLUTE_PATH_TO_deploy.cfg]
$ make deploy TARGET=[ABSOLUTE_PATH_TO_fakekb_DIR] DEPLOY_RUNTIME=[ABSOLUTE_PATH_TO_fakekb_DIR]
$ cd administration
$ python ./initialize.py
Keep this configuration? yes
Does mongodb require authentication? no
Please enter the name of your mongodb type database: ws_types
Choose a backend: g
$ cd ..
$ [PATH_TO_FAKE_KB]/services/workspace/start_service

Note

If workspace service does not start successfully, tail /var/log/syslog for errors.

  1. Check if the workspace service is working properly by creating a workspace service client, verifying workspace service version and creating a new workspace.

$ cd [PATH_TO_YOUR_WORKSPACE_DIR]/lib
$ ipython

In [1]: from biokbase.workspace.client import Workspace
In [2]: my_ci_token = 'YOUR CI TOKEN'
In [4]: ws = Workspace("http://localhost:7058", token=my_ci_token)
In [5]: ws.ver()
Out[5]: u'0.8.0-dev4'
In [6]: ws.create_workspace({'workspace': 'myws'})
Out[7]:
[1,
u'myws',
...
]