Ilya Sytchev, Harvard T.H. Chan School of Public Heath
David Jones, Sheffield Institute for Translational Neuroscience
Shannan Ho Sui, Harvard T.H. Chan School of Public Heath
Fritz Lekschas, Harvard Medical School
Jennifer Marx, Harvard Medical School
Scott Ouellette, Harvard Medical School
Winston Hide, Sheffield Institute for Translational Neuroscience
Peter Park, Harvard Medical School
Nils Gehlenborg, Harvard Medical School
The Stem Cell Commons
was developed by the Harvard Stem Cell Institute to create a community for stem cell bioinformatics. This open source environment for sharing and analyzing stem cell data combines genomics data sets with tools for discovery, analysis, visualization, and collaboration. The Commons uses the Refinery Platform
, an integrated web-based data analysis and visualization system, to enable reproducible analyses implemented as Galaxy workflows.
We originally deployed Refinery using instances of Galaxy on clusters in two different research computing facilities. However, limited control over scheduling, access, and deployment in these environments prevented us from moving from development to production. To allow for greater flexibility of the system, ensure reliability, optimize cost, scale based on demand, and facilitate collaboration, our recent efforts have focused on making Refinery easy to deploy in a cloud environment backed by Galaxy CloudMan. CloudMan
is a cloud manager that orchestrates the provision and management of Galaxy clusters on cloud infrastructures. Here we discuss our experiences of using key CloudMan features, such as automated deployment, cluster sharing, and autoscaling. We also describe some of the problems that we have encountered using CloudMan in novel and perhaps unanticipated ways.
Given our experience with Refinery and CloudMan so far, we believe that Galaxy can be deployed as a cloud-based analysis backend for other systems. We hope that by continued collaboration with the CloudMan and Galaxy developer communities, we can address the challenges that we are still facing.