Scientific Systems Software Developer

SLAC National Accelerator Laboratory is a U.S. Department of Energy laboratory operated by Stanford University. For over 60 years, SLAC has been at the forefront of scientific discovery, exploring how the universe works at the biggest, smallest, and fastest scales. From particle physics to astrophysics, materials science to biology, SLAC's world-class research facilities and scientific expertise drive innovation and push the boundaries of human knowledge.

Do you enjoy collaborating with a diverse group of people to solve complex challenges? Does contributing to breakthrough discoveries in science � and working in a world-leading research environment � excite you?

The Application and User Services (AUS) group is seeking an energetic, forward-thinking software engineer to develop tools and workflows for science projects such as the Vera C. Rubin Observatory US Data Facility, hosted at SLAC. You will work with a team that supports multipleglobal, open-science, collaborations eager to leverage cutting-edge, best-in-class computing, platform and data services. We continually explore new platforms and technologies, and you will have a direct hand in shaping what the future of scientific computing at SLAC looks like.

S3DF is SLAC's centralized scientific computing facility, providing unified high-performance computing, storage, and data services to a broad portfolio of science programs. Current users include the Vera C. Rubin Observatory US Data Facility (supporting the Legacy Survey of Space and Time), LCLS (the Linac Coherent Light Source X-ray free-electron laser), ATLAS and other HEP experiments, and a growing number of additional programs. These experiments collectively generate and manage data at petabyte scale, with demanding requirements for throughput, reliability, and scientific reproducibility.

You will design, implement, and support the applications and APIs used to process, manage, and serve scientific data across these programs. You will contribute to troubleshooting and tuning the full portfolio of services used by scientists worldwide. You will have experience  in data-intensive workflows and containerization and deployments on Kubernetes clusters. Some of your code will interface with authentication and authorization frameworks, so familiarity with Identity and Access Management (IAM) concepts is a plus.

A significant and growing part of this role will involve scientific data management at scale. The Facility manages a complex, multi-system data landscape � spanning distributed data repositories, metadata catalogs, and data access layers. You will help us bring coherence, reliability, and formal stewardship practices to this environment: aligning data lifecycle policy with operational reality, ensuring metadata integrity, and building the tooling and workflows that let us manage data responsibly across its full lifetime.

We encourage open dialog, free thinking, cooperation, and a growth mindset. This is an opportunity to learn, enable groundbreaking science, and develop your skills in a uniquely collaborative scientific computing environment.

• Design, implement, and maintain applications and APIs for S3DF science programs, enabling scientists worldwide to access, process, and analyze large-scale experimental and observational datasets
• Build and improve data-serving interfaces and processing pipelines that handle petabyte-scale astronomical datasets
• Containerize applications and deploy them on Kubernetes clusters, following modern cloud-native best practices
• Participate in design and development of software tools for scientific data management and data processing, with assignments varying according to experiment priorities and lifecycles.
• Integrate applications with Identity and Access Management frameworks (OIDC, SAML2, JWT, LDAP, COManage, etc.) to ensure secure and appropriate data access
• - Write clean, well-tested, well-documented code; contribute to and maintain shared software repositories across S3DF science programs

• Contribute to the integration and reconciliation of the Rubin data access layer with distributed data management systems (including Rucio), ensuring consistent views of datasets across systems
• Help develop and implement data lifecycle policies � defining how datasets are created, retained, migrated, and retired across storage tiers in alignment with scientific and operational requirements
• Work on tooling for metadata integrity and remediation: detecting inconsistencies between catalogs, registries, and physical storage, and building workflows to detect and resolve them
• Contribute to formalizing levels of data stewardship responsibility by dataset type � clarifying ownership, curation standards, and access controls across the Rubin data portfolio
• Develop and support workflows for exporting managed data subsets of data for downstream science, collaborators, and community data releases
• Help rationalize and consolidate the landscape of backing databases, improving consistency and reducing operational complexity
• Collaborate with Rubin data management teams across institutions to align on standards and tooling

• Support scientists and users in their day-to-day use of Data Facility services � diagnose issues, debug problems, and provide timely, clear resolutions
• Monitor the health and performance of deployed services and supporting infrastructure; gather metrics and produce reports
• Identify and resolve bottlenecks in data-intensive workflows, from ingestion and processing through serving and analysis
• Participate in on-call rotation and incident response for production services
• Develop and maintain runbooks, operational documentation, and user-facing guides
• Use the incident management system to track problems to resolution in a timely manner.

• Work closely with platform engineers to deploy and tune services on Kubernetes and HPC systems
• Contribute to CI/CD pipelines, GitOps workflows, and infrastructure-as-code tooling for Rubin services
• Collaborate with the broader scientific software community (including teams at partner institutions) to align on interfaces, standards, and shared components
• Evaluate and test emerging technologies and technical developments 
• Provide feedback and concrete recommendations for service improvements to the AUS team and scientific stakeholders

• Work effectively within a multidisciplinary team spanning software engineering, platform operations, and scientific research
• Communicate clearly with both technical peers and non-technical scientific users
• Contribute to team planning and agile/scrum processes
• Participate in cross-institutional software working groups and community forums

• Bachelor's degree in physics, computer science, or a related field, and 5 years of relevant experience in software development, systems administration, or scientific/high-performance computing or an equivalent combination of education and experience
• Proficiency in one or more programming languages; Python strongly preferred, with C/C++ or JavaScript a plus
• Experience with modern software development practices: version control (Git), CI/CD pipelines, code review, and agile/scrum methodologies
• Familiarity with data-driven and data-intensive workflows
• Hands-on experience with PostgreSQL (schema design, query optimization, and operational management)
• Experience with Apache Kafka or similar event streaming platforms for high-throughput, real-time data pipelines
• Practical experience deploying and operating applications on Kubernetes in a production environments at scale
• Understanding of distributed compute and storage systems, high-performance computing, and networking concepts
• Experience with system monitoring, benchmarking, and performance analysis
• Strong organizational and communication skills; ability to work effectively in a collaborative, distributed team environment
• Ability and genuine willingness to learn, adopt best practices, and grow technical skills on the job

• Experience with Python data science and scientific computing libraries: NumPy, Pandas, Jupyter, Bokeh, Parquet, Dask, and similar
• Production development and operational experience with container technologies (Docker, containerd, Singularity/Apptainer)
• Experience with configuration management and infrastructure-as-code tools: Ansible, GitHub Actions, GitLab CI, ArgoCD
• Familiarity with HPC job scheduling systems (SLURM, HTCondor)
• Experience with common infrastructure software: PostgreSQL/MySQL, nginx, Apache, Redis, or similar
• Knowledge of Identity and Access Management frameworks and technologies: OIDC, SAML2, JWT, Grouper, COManage, LDAP
• Exposure to astronomical data formats and tools (FITS, HDF5, Butler, Parquet-based catalogs) � or strong motivation to learn them quickly
• Familiarity with data management and cataloging systems used in large-scale science (e.g., Rucio, iRODS, or similar distributed data management platforms)
• Experience with data lifecycle management concepts: retention policies, tiered storage, provenance tracking, or data governance frameworks
• Experience working with or contributing to large open-source scientific software projects
• Familiarity with observability tooling: Prometheus, Grafana, Loki, or similar

• Effective Decisions: Uses job knowledge and sound judgment to make quality decisions in a timely manner
• Self-Development: Pursues a variety of venues and opportunities to continue learning and growing
• Dependability: Can be counted on to deliver results with a sense of personal responsibility for expected outcomes
• Initiative: Pursues work proactively with optimism, positive energy, and motivation to move things forward
• Adaptability: Flexes as needed when change occurs; maintains an open outlook while adjusting to new circumstances
• Communication: Ensures effective information flow to diverse audiences; creates and delivers clear, appropriate written and spoken messages
• Relationships: Builds relationships to foster trust, collaboration, and a positive climate in pursuit of common goals

• Be Part of ground-breaking science: Science at SLAC will transform our understanding of the universe � your work will be part of the infrastructure that makes it possible
• Real Ownership: You will own and operate production services used by scientists around the world
• Collaborative Science: Work embedded with a global scientific community tackling problems that matter

• Access to cutting-edge scientific computing infrastructure and a world-class research environment
• Support for attending relevant conferences and pursuing technical training
• Mentorship from experienced platform engineers and scientists
• Opportunities to grow into broader platform and architecture responsibilities over time

• Hybrid work arrangements possible
• State-of-the-art facilities on the Stanford University campus in the San Francisco Bay Area
• Collaborative team with a genuine commitment to work-life balance

• Competitive salary commensurate with experience
• Comprehensive health, dental, and vision insurance
• Retirement plans with employer contributions
• Generous vacation and paid time off
• Professional development funding
• Tuition reimbursement programs
• On-site amenities and wellness programs

• Consistent with its obligations under the law, the University will provide reasonable accommodation to any employee with a disability who requires accommodation to perform the essential functions of the job. May work extended hours during peak business cycles.
• Given the nature of this position, SLAC is open to on-site and hybrid work options.

• Interpersonal Skills: Demonstrates the ability to work well with Stanford colleagues and clients and with external organizations
• Promote Culture of Safety: Demonstrates commitment to personal responsibility and value for environment, safety, and security; communicates related concerns; uses and promotes safe behaviors based on training and lessons learned. Meets the applicable roles and responsibilities as described in the ESH Manual, Chapter 1 � General Policy and Responsibilities: http://www-group.slac.stanford.edu/esh/eshmanual/pdfs/ESHch01.pdf
• Subject to and expected to comply with all applicable University policies and procedures, including but not limited to the personnel policies and other policies found in the University's Administrative Guide: http://adminguide.stanford.edu
• As a national laboratory, SLAC National Accelerator Laboratory is responsible for adhering to the Homeland Security Presidential Directive 12 (HSPD-12) and Department of Energy (DOE) Order 473.1A, which require employees to obtain and maintain a HSPD-12 Personal Identity Verification (PIV) Credential. To obtain this credential, employees must successfully complete the applicable tier of federal background investigation post hire and receive a favorable federal adjudication.  The tier of federal background investigation will be determined by job duties and national security or public trust responsibilities associated with the job. All tiers of investigation include a declaration of illegal drug activities, including use, supply, possession, or manufacture within the last 1 to 7 years (depending on the applicable tier of investigation). Illegal drug activities include marijuana and cannabis derivatives, which are still considered illegal under federal law, regardless of state laws.

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

• Classification Title: Software Developer 2
• Grade: J, Job code: 4822
• Duration: Regular Continuing

The expected pay range for this position is $116,399 to $163,985 per annum. SLAC National Accelerator Laboratory/Stanford University provides pay ranges representing its good faith estimate of the salary the university reasonably expects to pay for a position upon hire. The pay offered to a selected candidate will be determined based on factors such as (but not limited to) the scope and responsibilities of the position, the qualifications of the selected candidate, departmental budget availability, internal equity, geographic location and external market pay for comparable jobs. At SLAC/Stanford, base pay represents only one aspect of the comprehensive rewards package.