Software Administration and Overrides

BioGrids provides flexible software management capabilities that allow site administrators to customize installations, override default versions, and manage software collections according to local requirements.

Overview

Software administration in BioGrids involves: - Version management: Control which software versions are available to users - Package overrides: Customize software configurations and dependencies - Collection management: Define custom software collections for specific research areas - Access control: Manage user access to different software packages - Local customizations: Add site-specific modifications and configurations

Version Override Management

Understanding Version Selection

BioGrids uses a hierarchical system for determining which software version to use:

1. User configuration (~/.biogrids.conf) - Highest priority
2. Site administrator overrides (/programs/admin/overrides.conf)
3. Default BioGrids version - Standard default from BioGrids catalog
4. Latest installed version - Fallback if no defaults specified

Site-Wide Version Overrides

Create site-wide version preferences in /programs/admin/overrides.conf:

# Example: /programs/admin/overrides.conf
# Force specific versions for all users

# Use specific BLAST version for consistency
BLAST_X=2.12.0

# Use older R version for compatibility with existing scripts  
R_X=4.1.2

# Use specific Python version
PYTHON_X=3.9.7

# Use CUDA-enabled version of specific software
RELION_X=3.1.1_cu9.2

# Use non-CUDA version for CPU-only machines
GROMACS_X=2022.4

Collection-Based Overrides

Define version overrides for specific software collections:

# Genomics collection overrides
[genomics]
SAMTOOLS_X=1.15
BWA_X=0.7.17
GATK_X=4.2.6

# Proteomics collection overrides  
[proteomics]
MAXQUANT_X=2.0.3
PROTEOME_DISCOVERER_X=2.4

# Structural biology collection overrides
[structural]
PYMOL_X=2.5.0
CHIMERA_X=1.4
COOT_X=0.9.8

Platform-Specific Overrides

Configure different versions for different platforms:

# Linux-specific overrides
[linux]
MATLAB_X=R2022a
SCHRODINGER_X=2022-1

# macOS-specific overrides  
[darwin]
MATLAB_X=R2021b  # Older version for macOS compatibility
SCHRODINGER_X=2021-4

Administrative Commands

Override Management

List Current Overrides

# Show all active overrides
biogrids-cli admin list-overrides

# Show overrides for specific software
biogrids-cli admin list-overrides blast

# Show collection-specific overrides
biogrids-cli admin list-overrides --collection genomics

Set Version Overrides

# Set site-wide version override
biogrids-cli admin set-override blast 2.12.0

# Set override for specific collection
biogrids-cli admin set-override --collection genomics samtools 1.15

# Set platform-specific override
biogrids-cli admin set-override --platform linux matlab R2022a

Remove Overrides

# Remove specific override
biogrids-cli admin remove-override blast

# Remove all overrides for software
biogrids-cli admin remove-override blast --all-versions

# Remove collection overrides
biogrids-cli admin remove-override --collection genomics samtools

Software Collection Management

Create Custom Collections

# Create new software collection
biogrids-cli admin create-collection "Metabolomics" \
    --software "xcms,metaboanalyst,mzmine,sirius" \
    --description "Metabolomics analysis software"

# Create collection from existing packages
biogrids-cli admin create-collection "RNA-Seq" \
    --from-installed "salmon,kallisto,rsem,star,hisat2"

Manage Collection Membership

# Add software to collection
biogrids-cli admin add-to-collection "Metabolomics" compound_discoverer

# Remove software from collection
biogrids-cli admin remove-from-collection "Metabolomics" old_software

# List collection contents
biogrids-cli admin list-collection "Metabolomics"

Package Configuration Management

Software Dependencies

# Override dependency requirements
biogrids-cli admin set-dependency relion openmpi@3.1.6

# Remove dependency requirement
biogrids-cli admin remove-dependency relion cuda

# List software dependencies
biogrids-cli admin list-dependencies relion

License Management

# Configure license server for commercial software
biogrids-cli admin set-license matlab \
    --server "license.university.edu" \
    --port "27000"

# Set license file path
biogrids-cli admin set-license schrodinger \
    --file "/programs/licenses/schrodinger.lic"

# List license configurations
biogrids-cli admin list-licenses

Advanced Administrative Features

Selective Software Installation

Install Specific Software Collections

# Install only genomics software
biogrids-cli admin install-collection genomics

# Install multiple collections
biogrids-cli admin install-collection genomics proteomics structural

# Exclude specific collections from installation
biogrids-cli admin --exclude-collection "Legacy Software" --all-versions

Version-Specific Installation

# Install only specific versions of software
biogrids-cli admin install-versions \
    blast@2.12.0 \
    samtools@1.15 \
    python@3.9.7

# Install latest versions of core software only
biogrids-cli admin install-core --latest-only

Environment Customization

Site-Specific Environment Variables

# Set site-wide environment variables
biogrids-cli admin set-env CUDA_VISIBLE_DEVICES "0,1"
biogrids-cli admin set-env OMP_NUM_THREADS "8"
biogrids-cli admin set-env TMPDIR "/local/scratch"

# Set software-specific environment
biogrids-cli admin set-env --software blast BLAST_DB_PATH "/data/blast_db"
biogrids-cli admin set-env --software matlab MLM_LICENSE_FILE "27000@license.server"

Custom Module Files

# Generate module files for software
biogrids-cli admin generate-modules --output /programs/modulefiles

# Create custom module file
biogrids-cli admin create-module blast \
    --version 2.12.0 \
    --environment "BLAST_DB_PATH=/data/blast_db" \
    --prereq "gcc/9.3.0"

User Access Control

Software Access Restrictions

# Restrict access to commercial software
biogrids-cli admin restrict-access matlab --group matlab_users
biogrids-cli admin restrict-access schrodinger --group schrodinger_users

# Allow access to specific software
biogrids-cli admin allow-access --user researcher1 --software "matlab,schrodinger"

# List access restrictions
biogrids-cli admin list-restrictions

Usage Monitoring

# Enable usage tracking
biogrids-cli admin enable-tracking --software matlab,schrodinger

# View usage statistics
biogrids-cli admin usage-stats --software matlab --period "last-month"

# Generate usage reports
biogrids-cli admin generate-report --output /tmp/usage-report.html

Configuration Files

Main Override Configuration

The primary override configuration file is located at /programs/admin/overrides.conf:

# Global BioGrids Administrator Overrides Configuration
# This file controls site-wide software version preferences

[global]
# Site-wide version preferences
PYTHON_X=3.9.7
R_X=4.2.1
JAVA_X=11.0.2

[genomics]
# Genomics-specific overrides
BLAST_X=2.12.0
SAMTOOLS_X=1.15
BWA_X=0.7.17
GATK_X=4.2.6
STAR_X=2.7.9a

[proteomics] 
# Proteomics-specific overrides
MAXQUANT_X=2.0.3
MASCOT_X=2.8
PROTEOME_DISCOVERER_X=2.4

[structural]
# Structural biology overrides
PYMOL_X=2.5.0
CHIMERA_X=1.4
COOT_X=0.9.8
PHENIX_X=1.20.1

[machine_learning]
# ML/AI software overrides
TENSORFLOW_X=2.8.0
PYTORCH_X=1.11.0
JUPYTER_X=6.4.0

[development]
# Development environment overrides
GCC_X=11.2.0
CMAKE_X=3.22.1
GIT_X=2.35.1

Collection Configuration

Define custom software collections in /programs/admin/collections.conf:

# Custom Software Collections Configuration

metabolomics:
  name: "Metabolomics Analysis"
  description: "Software for metabolomics data analysis"
  software:
    - xcms
    - metaboanalyst  
    - mzmine
    - sirius
    - compound_discoverer
  dependencies:
    - r
    - java
    - python

single_cell:
  name: "Single Cell Analysis"
  description: "Tools for single-cell genomics and transcriptomics"
  software:
    - cellranger
    - seurat
    - scanpy
    - monocle3
    - velocyto
  dependencies:
    - r
    - python
    - samtools

imaging:
  name: "Biological Image Analysis" 
  description: "Image analysis and microscopy software"
  software:
    - imagej
    - fiji
    - cellprofiler
    - ilastik
    - napari
  dependencies:
    - java
    - python

License Configuration

Configure software licenses in /programs/admin/licenses.conf:

[matlab]
license_type = "server"
license_server = "license.university.edu"
license_port = "27000"
feature_name = "MATLAB"

[schrodinger]
license_type = "file"
license_file = "/programs/licenses/schrodinger.lic" 
license_host = "license.university.edu"

[gaussian]
license_type = "file"
license_file = "/programs/licenses/gaussian.lic"
scratch_dir = "/local/scratch/gaussian"

[vmd]
license_type = "free"
registration_required = true

Monitoring and Maintenance

Administrative Monitoring

System Health Checks

# Check override configuration validity
biogrids-cli admin validate-config

# Verify software dependencies
biogrids-cli admin check-dependencies

# Test license servers
biogrids-cli admin test-licenses

Usage Analytics

# Generate usage reports
biogrids-cli admin usage-report --format html --output /var/www/biogrids-usage.html

# Monitor software access patterns
biogrids-cli admin monitor-access --real-time

# Export usage data
biogrids-cli admin export-usage --format csv --period "last-quarter"

Automated Administration

Scheduled Maintenance

# Add administrative tasks to cron
# Check configuration daily
0 6 * * * /programs/biogrids-cli admin validate-config --email-report

# Update usage statistics weekly  
0 3 * * 1 /programs/biogrids-cli admin update-usage-stats

# Generate monthly reports
0 2 1 * * /programs/biogrids-cli admin generate-report --monthly

Automated Overrides

# Script to automatically set overrides based on policies
#!/bin/bash
# auto-override.sh

# Set overrides for new software versions after testing period
biogrids-cli list --updates | while read software version; do
    # Wait 30 days before making new versions default
    release_date=$(biogrids-cli info $software@$version | grep "Release Date")
    if [[ $release_date < $(date -d "30 days ago") ]]; then
        biogrids-cli admin set-override $software $version
    fi
done

Best Practices

Version Management Strategy

1. Stability over novelty: Prefer well-tested versions over latest releases
2. Consistency: Maintain consistent versions across related software
3. Documentation: Document all overrides with reasons and dates
4. Testing: Test override changes in development environment first
5. Communication: Notify users of version changes in advance

Security Considerations

1. Access control: Implement proper file permissions for override configurations
2. License compliance: Ensure all license requirements are met
3. Audit trail: Maintain logs of all administrative changes
4. Backup: Regular backup of override and configuration files

Performance Optimization

1. Selective installation: Install only needed software collections
2. Version cleanup: Regularly remove unused old versions
3. Dependency optimization: Minimize unnecessary dependencies
4. Resource allocation: Configure appropriate resource limits

For additional guidance on software administration, contact help@biogrids.org with specific questions about your site configuration requirements.