XSEDE16

Rapid secure data sharing and private online discussion are requirements for coordinating today’s distributed science teams using High Performance Computing (HPC), visualization, and complex workflows. Modern HPC infrastructures do a good job of enabling fast computation, but the data produced remains within a site’s storage and network environment tuned for performance rather than broad easy access. To share data and visualizations among distributed collaborators, manual efforts are required to move data out of HPC environments, stage data locally, bundle data with metadata and descriptions, manage versions, build animations, encode videos, and finally post it all online somewhere for secure access and discussion among project colleagues. While some of these tasks can be scripted, the effort remains cumbersome, time-consuming, and error prone. A more streamlined approach is needed.
In this paper we describe SeedMe – the Stream Encode Explore and Disseminate My Experiments platform for web-based scientific data sharing and discussion. SeedMe provides streamlined and application-controlled automatic data movement from HPC and desktop environments, metadata management, data descriptions, video encoding, secure data sharing, threaded discussion, and, optionally, public access for education and outreach.

As the computational movement gains more traction in the scientific community, there is an increasing need to understand what drives adoption and diffusion of tools. This investigation reveals what makes a computational tool more easily adopted by users within the e-science community. Guided by Rogers’s [1] Diffusion of Innovations theory, we set out to identify the innovation attributes of a range of computational tools across domains. Based on 135 interviews with domain scientists, computational technologists, and supercomputer center administrators across the U.S. and a small portion from Europe, systematic analysis revealed 10 key attributes. They are: driven by needs, organized access, trialability, observability, relative advantage, simplicity, compatibility, community-driven, well-documented, and adaptability. We discuss the attributes in the form of questions stakeholders should keep in mind while designing and promoting the tools. We also present strategies associated with each attribute. The 10 attributes and associated questions can serve as a checklist for e-science projects that aim to promote their computation tools beyond the incubators. This paper is submitted to the "Software and Software Environments" track because it has implications for engagement of user communities.

HPC centers run a diverse set of applications from a variety of scientific domains. Every application has different resource requirements, but it is difficult for domain experts to find out what these requirements are and how they impact performance. In particular, the utilization of shared resources such as parallel file systems may influence application performance in significant ways that are not always obvious to the user. We present a tool, REMORA, that is designed to provide the information that is most critical for running efficiently an application in HPC systems. REMORA collects runtime resource utilization data for a particular job execution and presents a user-friendly summary on completion. The information provided forms a complete view of the application interaction with the system resources, which is typically missing from other profiling and analysis tools. Setting up and running REMORA requires trivial effort, and can be done as a regular user with no special permissions. This enables both users and administrators to download the tool and identify a particular application’s resource requirements within minutes, helping in the diagnosis of errors and performance issues. REMORA is designed to be scalable and have minimal impact on application performance, and includes support for NVIDIA GPUs and Intel Xeon Phi coprocessors. It is open source, modular, and easy to modify to target a large number of HPC resources.