Senior HPC Administrator, Secure HPC Platform - High Performance Computational and Data Ecosystem - Technology Specialist III - Hybrid/Onsite

The Scientific Computing and Data group at the Icahn School of Medicine at Mount Sinai partners with scientists to accelerate scientific discovery. To achieve these aims, we support a cutting-edge high-performance computing and data ecosystem along with MD/PhD-level support for researchers. The group is composed of a high-performance computing team, a clinical data warehouse team and a data services team.

The Senior HPC Administrator, High Performance Computational and Data Ecosystem is responsible for the operation, security, and governance of a secure HPC environment supporting multiple regulated research and clinical workloads. These include CMS (Centers for Medicare & Medicaid Services) controlled data subject to NIST SP 800‑53 (Data Management Self-Attestation Questionnaire - DMP SAQ) and NIH‑controlled genomic datasets subject to NIST SP 800‑171, and equivalent regulatory frameworks. This role ensures continuous compliance with federal, institutional, and sponsor cybersecurity and data governance requirements while delivering high‑performance, reliable, and auditable research computing services.

To support Sinai’s scientific and clinical mission, the Senior HPC Administrator brings deep technical expertise in computational systems, data platforms, and research software environments, with a strong emphasis on cybersecurity, compliance, and researcher‑focused service delivery. The role requires independent leadership, advanced troubleshooting skills, and the ability to drive complex initiatives to completion with minimal supervision.

Scientific Computing operates a large‑scale computational and data ecosystem, including a 30,000+ core HPC cluster and 50+ PB of usable storage, as well as a clinical data warehouse and secure software development environments. The HPC system is ranked among the fastest academic biomedical computing platforms globally (Top500). 

This position reports to the Senior Director for Computational & Data Ecosystem in Scientific Computing. Specific responsibilities are listed below.

HPC Administration, and HPC Cybersecurity 

• Provide full lifecycle administration of secure HPC infrastructure, including compute, storage, networking, and supporting services. Manage system configuration, authentication and authorization, monitoring, logging, performance tuning and analytics, usage reporting, and capacity planning.
• Administer and support core HPC services such as job scheduling platforms (e.g., LSF), Open OnDemand user access environments, and PostgreSQL databases.
• Design, maintain, and enforce secure network architectures, including segmentation, firewall rules, and controlled ingress/egress.
• Lead the troubleshooting, isolation and resolution of all technical issues including system, hardware, software, and network. 
• Implement and maintain NIST‑aligned security controls, including encryption of data at rest and in transit, secure enclaves, access control, centralized logging, audit trails, and monitoring. Lead system security hardening, patching, vulnerability management, and technical risk remediation activities.
• Ensure continuous compliance with CMS DMP SAQ requirements, NIST SP 800‑171/800‑53 controls, NIH data governance policies, and Mount Sinai cybersecurity standards.
• Develop, maintain, and regularly update security policies, operational procedures, and technical documentation supporting regulatory compliance.
• Serve as the primary technical and process liaison with DTP Cybersecurity, GCO, and compliance stakeholders. Coordinate incident response activities and security investigations in partnership with DTP Cybersecurity.
• Organize and manage annual external audits and recertification efforts, ensuring timely remediation and sustained audit readiness.
• Stay current with emerging HPC and cybersecurity technologies to keep the organization’s infrastructure up-to-date. 
• Work effectively and productively with other team members within the group and across Mount Sinai. 
• Support researchers and users with computational, data, and analytics needs, including data transfers, software installation, batch jobs and troubleshooting etc. Answers and resolves user tickets. Create and update user-facing technical documentation for the CMS cluster.
• Provide after-hours support as needed. 
• Perform other duties as assigned or requested.

• Bachelors degree in a technical discipline in computer science, engineering or related scientific field. Master's or PhD preferred
• 8 years minimum experience (higher preferred), 12 years of related experience, including a demonstrated ability in technology area. In-depth knowledge of associated technology areas that could impact area of responsibility; healthcare technology experience preferred.

Specific experience for this position:

• 8+ years of progressive HPC systems administration experience in large‑scale Red Hat/CentOS Linux batch environments.
• Hands-on experience with HPC schedulers, parallel file systems, configuration management (e.g., xCAT, Puppet, Ansible), databases, web services, globus and high‑performance networking (InfiniBand, Ethernet).
• Must be an expert troubleshooter; Must be a team player and customer focused
• Strong background in cybersecurity practices within regulated environments (e.g., healthcare, government), including firewalls, data protection, access controls, monitoring, and incident response.
• In‑depth knowledge of NIST SP 800‑171 and NIST SP 800‑53, with hands‑on experience implementing, and operating compliant HPC systems.
• Script and programming experience
• Ability to manage multiple priorities effectively in a dynamic environment
• Excellent analytical ability, strong judgment and management skills, and the ability to work effectively as a liaison between both research and technology teams.
• Excellent written, verbal, and interpersonal communication skills.
   

Preferred Experience

• Experience with IBM LSF and GPFS Spectrum Scale.

• Experience in an academic or research environment.

• Experience with confidential computing and privacy-preserving linking of PHI.

• Experience with public cloud platforms (Azure, AWS, or GCP).

Compensation Statement

The Mount Sinai Health System (MSHS) provides salary ranges that comply with the New York City Law on Salary Transparency in Job Advertisements. The salary range for the role is $113,011.58 - $202,155.00 Annually. Actual salaries depend on a variety of factors, including experience, education, and operational need. The salary range or contractual rate listed does not include bonuses/incentive, differential pay or other forms of compensation or benefits.

Non-Bargaining Unit, 297 - DTP Infrastructure Planning - MSH, Mount Sinai Hospital